Shelteredchap3

Sheltered vs. Supported Employment:
A Direct Comparison of Long-Term
Earnings Outcomes for Individuals
with Cognitive Disabilities
John Kregel and David H. Dean
The long-term earnings impact of sheltered and supported employment on 877 individualswith cognitive disabilities was investigated through the implementation of a comprehensiveanalytical framework for assessing employment outcomes for people with disabilities whohave been served by a vocational rehabilitation (VR) agency in a single state. Informationon demographic characteristics, pre- and post-program earnings, and local economicconditions were merged to allow a comprehensive examination of the earnings outcomesof individuals who receive alternative types of VR services. Results indicated that peopleserved in sheltered and supported employment differ in many ways from other individualswith cognitive disabilities successfully served by the VR agency. Individuals in the supportedemployment cohort were more likely to have worked in competitive, integrated employmentprior to program entry than their sheltered employment counterparts. The sheltered andsupported employment cohorts differ slightly in terms of demographic characteristics.
Earnings of the supported employment cohort were 250% greater than the shelteredemployment cohort across a seven-year post-program period. A fixed effects modeling(FEM) procedure revealed that supported employment had a statistically greater impact onthe earnings of persons with disabilities than participation in sheltered employment.
Implications are discussed in relation to the recent U.S. Department of Education decisionto amend the definition of employment outcome.
On January 22, 2001, the U.S. Department of Education published final regulations governing the State Vocational Rehabilitation Services Program that amended the definition of “employment outcome” (Federal Register,January 22, 2001). The revised regulations overturned a decades old policy that allowed State Vocational Rehabilita-tion (VR) agencies to consider placement in segregated employment settings such as sheltered workshops to be anappropriate employment outcome for individuals with disabilities. Under the amended regulations, employmentoutcome is defined as full or part-time employment in the integrated labor market. Appropriate employment out-comes also include supported employment, self-employment, telecommuting, or business ownership.
The amended regulations have been controversial. The Department of Education received over 3,000 comments in response to the initial notice of proposed rulemaking in June, 2000. Proponents of the amendedregulations contended that the change would significantly increase access to integrated employment settings,eliminate barriers to competitive jobs for persons with significant disabilities, and improve the overall accountability ofthe VR program. Critics expressed concern that restricting the definition of appropriate employment outcome exclu- sively to integrated employment settings would unnecessarily limit consumer choice, reduce the number of employ-ment options available to individuals with disabilities, limit access to the VR system for individuals with the mostsignificant support needs, and lead to the elimination of sheltered workshops where individuals currently work.
Sheltered employment programs are designed to assist individuals who for whatever reason are viewed as not capable of working in a competitive employment setting in their local community. The term “sheltered employ-ment” is often used to refer to a wide range of segregated vocational and non-vocational programs for individualswith disabilities, such as sheltered workshops, adult activity centers, work activity centers, and day treatment centers.
These programs differ extensively in terms of their mission, services provided, and funding sources. Currently, mostsheltered employment services are operated through private, not-for-profit organizations that are funded through avariety of State and Federal funding sources.
Virtually all forms of sheltered employment can generally be classified into two types. Transitional employ- ment programs are intended to provide training and experience to individuals in segregated settings so that they willbe able to acquire the skills necessary to succeed in subsequent competitive employment. Extended employmentprograms are designed to be long-term or permanent placements for individuals that will allow them to use theirexisting abilities to earn wages in the segregated workshop setting.
In amending the regulations regarding employment outcome, the Department of Education recognized the two different types of sheltered employment settings. “Extended employment (i.e. sheltered or non-integratedemployment) remains both an initial step toward achieving integrated employment under the VR program and a long-term employment option through sources of support other than the VR program” (Federal Register, January 22, 2001,p. 7254). In other words, the revised regulations still recognize the transitional (training) function of sheltered work-shops by allowing the use of VR funds to support an individual’s placement in a sheltered workshop, if the purposeof that placement is to prepare for work in an integrated employment setting. Furthermore, the regulations acknowl-edge that placement sheltered employment may still remain a long-term outcome for individuals with disabilities. Theregulations merely require that funds other than VR monies be used to support this segregated placement.
The move by the Department of Education to focus the use of VR funds on the achievement of integrated employment outcomes is the most recent episode in a long and contentious debate over the appropriateness ofsheltered employment for persons with cognitive disabilities (McLoughlin, Garner, & Callahan, 1987; TenBroek &Matson, 1959; Wehman, 1981; Whitehead, 1979). Sheltered workshops and other segregated employment settingsformed the core of our nation’s system of vocational training and employment supports for adults with significantcognitive disabilities throughout the twentieth century. The majority of adults with cognitive disabilities still work insegregated, sheltered employment settings, as opposed to integrated settings in the competitive labor force.
Despite the continued reliance on these programs, critics have consistently questioned the effectiveness of segre-gated employment settings. For example: Sheltered employment settings fail to provide individuals meaningful employmentoutcomes. Earnings are low or inconsequential, forcing individuals to remain depen-dent on Federal cash benefit programs (Bellamy, Rhodes, Bourbeau, & Mank, 1986).
Sheltered employment programs unnecessarily isolate individuals from the rest oftheir community. Rather than lessening obstacles to employment for persons with (continued)
disabilities, this segregation actually contributes to lowered expectations andnegative public attitudes (Wehman, 1981).
Once in sheltered employment, very few persons are able to progress into competi-tive employment. The long-term impact of sheltered employment on the productivityand community integration of individuals with disabilities is very small (Murphy &Rogan, 1995).
In reaction to these criticisms, supported employment has emerged as an alternative to sheltered employ- ment, particularly for individuals with cognitive disabilities. Supported employment, developed in the Federal/StateVR program in 1986, is viewed as competitive employment in an integrated work setting for individuals traditionallyunable to obtain and maintain employment in the open labor market. As recently as 1986, only 300 programs acrossthe country provided supported employment services. Today, over 3,000 agencies provide supported employmentto over 250,000 individuals across the nation (Kregel & Wehman, 1997).
Despite its rapid emergence as the preferred employment alternative for individuals with significant cognitive disabilities, supported employment has also been occasionally criticized (Kregel &Wehman, 1989; Wehman &Kregel, 1995). Concerns have been raised that individuals with cognitive disabilities are frequently unable to retaintheir jobs for extended periods of time after initial placement, individuals are often placed into low-paying jobs that tonot allow them to achieve economic self-sufficiency, and the program fails to effectively meets the needs of individu-als with the most significant and ongoing support needs.
The ability of both sheltered and supported employment to have a long-term impact on the earnings and economic self-sufficiency of individuals with cognitive disabilities directly relates to the Department of Education’sdecision to modify the definition of employment outcome. The revised regulations reject sheltered employment as ameaningful employment outcome for the VR program, yet retain sheltered employment as in “interim step in therehabilitation process” (p. 7255).
While the relative merits of sheltered and supported employment have been widely debated, little empirical evidence exists that directly compares the outcomes generated by these programs for individuals with cognitivedisabilities (Cimera, 1998;. Coker, Osgood, & Clouse, 1995). Existing research has been hampered by a lack ofvalid comparisons between sheltered employment and supported employment populations, lack of long-termearnings data, especially for individuals who have exited sheltered employment settings, and an inability to accountfor the impact of changing economic conditions on earnings outcomes. The present study attempts to avoid some ofthe shortcomings of previous research through the implementation of a comprehensive analytical framework forassessing employment outcomes for persons with disabilities who have been served by a VR agency in a singlestate.
The purpose of this investigation was to compare the characteristics and long-term employment outcomes of individuals with cognitive disabilities placed into sheltered employment or supported employment by a state VRagency. Specifically, the study investigates the following questions: How do individuals placed into sheltered or supported employment differ from otherindividuals with cognitive disabilities served by the state vocational rehabilitationagency? (continued)
What are the demographic characteristics and pre-employment earnings histories ofindividuals with cognitive disabilities placed into sheltered or supportedemployment? What is the long-term earnings impact of placement into sheltered or supportedemployment on persons with cognitive disabilities served by a state vocationalrehabilitation agency? The present study is a part of a larger effort to conduct a long-term evaluation of the federal-state vocational rehabilitation (VR) program in a single state (Virginia). The overall effort attempts to develop and pilot a comprehen-sive framework to evaluate the employment outcomes achieved by individuals who receive various types of ser-vices through the state VR agency (Dean et al., 2001). Key features of the analytical framework include: Development of alternative “internal” comparison groups, including program dropoutsas a method of controlling for selection bias; Development of an extensive earnings profile for each applicant that contains threeyears of pre-VR earnings and seven years of post-VR earnings; Measures of the local business climate that might influence employmentopportunities for individual clients; and Use of a “fixed effects modeling” (FEM) analysis to examine the impact of vocationalrehabilitation services on post-program earnings that controls for the influence ofdemographic characteristics, socio-economic factors, and local economicconditions.
The larger study looks at the longitudinal earnings profiles for over 4,000 applicants for VR services in Virginia in 1988, including individuals with differing types of disabilities who received a wide variety of servicesthrough the federal state VR program. The present study focuses only on one group of individuals (people withcognitive disabilities) who were closed into sheltered or supported employment situations.
Data Sources
Three distinct types of data, each derived from a different administrative data set, were merged to compare the experiences of individuals with cognitive disabilities served by vocational rehabilitation in sheltered and sup-ported employment programs. These data included information on the demographic characteristics of individualsapplying for services, indicators of local economic conditions, and pre-service and post-service individual earningshistories for all sample participants. Each of these data types is described on the following page.
Demographic and Socio-Economic Characteristics
Information on the individual’s age, sex, disabling condition, ethnicity, educa-tional experiences, benefits status and other relevant information was obtainedfrom the federally mandated reporting system (RSA 911) maintained by the stateVR agency.
Local Economic Conditions
Annual information on per capita income and unemployment rates was obtainedfor each of the state’s 21 “planning districts” for each year from 1985 to 1997, inorder assess the impact of local economic factors on earnings outcomes acrossdiverse local communities.
Earnings Histories
The major dependent variable for this study was the annualized earnings ofprogram applicants ultimately served in sheltered and supported employmentsettings. These data are obtained through the transfer of a series of administra-tive databases provided by the Virginia Employment Commission (VEC). Thecombined VEC database contains quarterly earnings throughout the period1985-1997. For the individuals in the present sample, these data allow thedevelopment of an “earnings profile” that describes each individual’s earningshistory during the three years prior to receiving VR services and the seven yearsafter receiving services.
The sample for the investigation consisted of 877 individuals with (1) a diagnostic label of cognitive disability (mental retardation), (2) who received services through the public vocational rehabilitation program in Virginia, and (3)who were closed into sheltered (N=447) or supported (N = 430) employment status. Individuals who were placed intoboth sheltered and supported employment were excluded from the sample. To create a sufficient sample for theanalyses, the sample is comprised of individuals who were either closed into sheltered or supported employmentduring federal Fiscal Year (FY) 1988, or who applied for VR services in 1988 and were closed into sheltered orsupported employment in subsequent years.
Since the applicants and closure groups were accepted under basically the same eligibility criteria and were receiving VR services at roughly the same point in time, it seems appropriate to combine for analyses. Nonethe-less, to determine if significant differences exist between the two cohorts, a battery of “difference in means” tests wasconducted on multiple demographic, service and outcome variables (Dean, Dolan, & Schmidt, 2001). Resultsshowed only one statistically significant difference between the closure and applicant (i.e. the likelihood of benefitsreceipt at the time of application). No other statistically significant demographic differences were detected. While it isimportant to acknowledge these difference, the closure and applicant cohort were remarkably similar in over 10demographic and program earnings variables. On balance, merging the two cohorts appears to be justified and thetwo cohorts will be reflected in all reported data.
Comparison of the Sheltered Employment and Supported Employment Groups
with Other Individuals Served by the VR Agency
An initial set of analyses was performed to examine the demographic and socio-economic characteristics of the sheltered and supported employment groups to other individuals with cognitive disabilities who (1) were included in the same application/closure cohort, (2) did not receive any type of sheltered employment or supportedemployment service, and (3) were successfully served by the VR agency (closed into status 26 – SuccessfullyRehabilitated). These analyses were completed to determine whether the sheltered and supported employmentgroups differed from the “mainstream” vocational rehabilitation population in ways that would justify their service inthese specialized programs.
Table 1 below summarizes the demographic characteristics for the sheltered, supported, and status 26 groups. As indicated in the table, the sheltered and supported employment groups differed from the Status 26 groupin a number of ways.
Individuals in the sheltered and supported employment groups were significantlymore likely to have been classified with a moderate or severe cognitive disability, asopposed to a classification of mild cognitive disability.
Females comprised a slightly larger percentage of the sheltered and supportedgroups, as opposed to the Status 26 group.
African-Americans comprised nearly half (46%) of the Status 26 group, as comparedto 29% of the sheltered employment and 36% of the supported employment groups.
Individuals in the sheltered and supported employment programs were five timesmore likely to receive Social Security Disability Insurance and/or SupplementalSecurity Income than persons in the Status 26 group.
In summary, individuals in the sheltered and supported employment groups appear to possess more significant support needs, as indicated by their diagnostic classification and their dependence on SSA benefitprograms, in comparison to the Status 26 group. The Status 26 group was more likely to be African-American andmale. The sheltered and supported employment groups were more like to be white and female.
Table 1 -- Demographic and Socio-Economic Characteristics of the
Sheltered Employment, Supported Employment and Status 26 Groups
Sheltered
Supported
Status 26
Characteristic
Employment
Employment
Comparison of the Pre-Program Earning and Demographic Characteristics of the Sheltered
Employment and Supported Employment Groups
Information on pre-program earnings of individuals in the sheltered and supported employment cohorts was obtained from the longitudinal earnings data provided by the Virginia Employment Commission (VEC). The demo-graphic and socioeconomic attributes available from the federal RSA 911 reporting form include the specific disabilityclassification, gender, race/ethnicity, and whether the person was receiving a disability-related federal transferpayment such as Disability Insurance (DI) or Supplemental Security Income (SSI). Mean values for the sheltered andsupported employment groups on selected variables are contained in Table 2. A brief discussion of each character-istic will be presented along with the results of a test detecting a statistically significant difference in means. Whenappropriate, results will be stratified by the level of cognitive disability (i.e., mild, moderate, or severe).
Pre-Program Earnings
People in sheltered and supported employment were first compared on pre-program earnings in order to assess their employment experiences prior to entering the VR program. Difference-in-means tests were performedto compare pre-program earnings three years, two years, and one year prior to entry into the VR program. Theresults are summarized in Table 2. The two entries for each of the three earnings periods provided in rows 1-3 arethe percentage of individuals with earnings reported to the Virginia Employment Commission, and then the averageannual earnings for only those with some (non-zero, or “>0 Only”) earnings in the period. The distinction is importantbecause while there are no statistically significant differences in any of the three pre-VR years using the “non-zero”mean value, differences were detected in the percentage of individuals with earnings reported to VEC. It should beemphasized that earnings from sheltered employment would not be reported to VEC. Therefore, the percentages inRows 1-3 should be viewed as the percentage of individuals in each cells with earnings from competitive, integratedemployment in jobs “covered” by VEC.
Statistically significant differences were found when earnings for all participants are compared for the three year (t-value = 1.95, p < .05), two year (t-value = 3.81, p < .01), and one year (t-value = 2.01, p < .05) time periods. Inother words, persons in supported employment were statistically more likely to earn more in each of the three yearsprior to enrollment in vocational rehabilitation than their counterparts in sheltered employment. However, thisdifference is primarily caused by the fact that a larger percentage of individuals in sheltered employment had noearnings in one or more of the pre-program years. In instances where individuals in sheltered employment had VECreported pre-program earnings, those earnings were statistically equivalent to the supported employment cohort.
Diagnostic Classifications
As indicated at the bottom of Table 2 on the following page, a Chi-square test of difference in proportions for the three cognitive disability diagnostic classification levels (mild, moderate, severe) reveals that there is a statisti-cally significant difference across the two cohorts. While both groups had an equal number of individuals in themoderate diagnostic classification, members of the supported employment group were more likely to be classifiedwith a mild cognitive disability and individuals in the sheltered employment group more likely to classified with asevere cognitive disability. While the results were statistically significant, it should be noted that there was consider-able overlap between the two groups. For example, over 21% of the supported employment group were classifiedwith a severe cognitive disability, 18% of the sheltered employment group were classified with a mild cognitivedisability, and over 50% of both groups were classified with a moderate cognitive disability.
Table 2 -- Difference-in-Means Tests
Comparison of Sheltered and Supported Employment Participants
Variable
* Significant at 5%, two-tailed. ** Significant at 1%, two-tailed.
Gender Composition
A test of the difference-in-means for the proportion of the overall sheltered and supported employment cohorts that were male or female found no significant difference between the two groups. However, when the twocohorts are stratified by the diagnostic classification level (mild, moderate, severe), differences are uncovered (Table3 below). Difference-in-means tests for each of the mild, moderate, and severe classification levels reveal a statisti-cally significant difference in the proportion that are male for the mild classification level cohorts located in supported(60%) versus sheltered (45%) employment (t-value = 2.06, p < .05).
Table 3 -- Gender of Sheltered and Supported Cohorts,
Sorted by Diagnostic Classification
Supported Employment
Sheltered Employment
Moderate
Moderate
Racial and Ethnic Background
As described in the previous section, persons of diverse ethnic backgrounds participate less often in both sheltered and supported employment in comparison to their rate of participation in other programs operated by thevocational rehabilitation agency. Table 4 below examines the racial and ethnic backgrounds of the sheltered andsupported employment cohorts stratified by diagnostic classification. The participation rate of African Americanssteadily increases in each cohort through the mild, moderate and severe subgroups. While almost three-fourths ofpersons in supported employment with mild cognitive disabilities are white, the figure drops to slightly more thanone-half for persons with severe cognitive disabilities. A similar pattern in found across the three subgroups of thesheltered employment cohort. Difference in means tests found significant differences only between the two sub-groups of persons with severe cognitive disabilities. African-Americans were represented at a significantly higherrate in the supported employment group of persons with severe cognitive disabilities, as compared their counter-parts in sheltered employment (t value = -2.00, p < .05 two tailed).
Table 4 -- Racial/Ethnic Background for Supported vs. Sheltered
Employment Cohorts, Sorted by Diagnostic Classification
Supported Employment
Sheltered Employment
Race/Ethnic
Background
Moderate
Moderate
Receipt of Transfer Payment Income
People receiving sheltered employment or supported employment services are far more like to be recipi- ents of SSDI or SSI than individuals participating in other vocational rehabilitation programs. As indicated in Table 2,difference of means tests conducted on the two cohorts found that persons in sheltered employment are statisticallymore likely to receive SSI benefits than supported employment participants; no similar difference was found in thedirect comparison of the cohorts based on SSDI benefit status. Table 5 on the following page presents the percent-age of individuals in each diagnostic level for both groups who received SSI and SSDI. Individuals in both groupsare far more likely to receive SSI payments, as opposed to SSDI benefits.
As might be expected, persons with mild cognitive disabilities are generally less likely to receive SSI benefits than members of other subgroups. Individuals with severe cognitive disabilities are most likely to receiveSSI benefits. The SSI-receipt rates are lower for the cohorts with mild and moderate cognitive disabilities in sup-ported employment, compared to their counterparts receiving sheltered employment, 33.6% vs. 41.3%, and 45.8 vs.
57.1%, respectively. Difference in mean tests reveals that these variations are statistically significant for the groupwith moderate cognitive disabilities (t-value = -2.56, p < .05 two-tailed). Conversely, the receipt rate is higher, 73.2 vs. 64.2%, for the supported employment cohort with severe cognitive disabilities than for their counterparts insheltered employment, although this difference was not found to be statistically significant.
The SSDI receipt status for the sheltered and supported subgroups, stratified by diagnostic classification, is also provided in Table 5. As with SSI, the SSDI-benefit receipt rate increases across the mild, moderate, and severecognitive disabilities subgroups. Nearly one in four individuals diagnosed with severe cognitive disabilities andreceiving supported employment receive SSDI; for comparable individuals receiving sheltered employment, theSSDI receipt rate is one in five. The receipt rate is 13.6% for the supported employment cohort with mild cognitivedisabilities compared to only 8.7% for the comparable cohort receiving sheltered employment. The difference-in-means tests comparing these various cohorts did not detect statistically significant differences.
Table 5 -- SSI and SSDI Benefits Status, Sorted by Diagnostic
Classification
Percentage Receiving SSI
Percentage Receiving SSDI
Employment
Moderate
Moderate
An outcomes-based analysis framework was used to compare the difference in earnings from a pre-VR and a post-VR period for the sheltered and supported employment cohorts. This type of analysis requires a stream oflongitudinal earnings over multiple years. Moreover, the pre-vocational rehabilitation earnings streams for bothgroups are required to conduct the statistical tests to validate a potential comparison.
The first step in constructing pre- and post-program earnings streams is determining whether each sample member has the necessary earnings that would allow the person to be included in the analysis. The earnings profileis developed from quarterly earnings from the Virginia Employment Commission (VEC). The percentage of individu-als reporting earnings rates are provided in Table 6 for the sheltered and supported employment cohorts, stratifiedby level of cognitive disability. As might be expected, both cohorts with severe cognitive disabilities had the lowestrates of reported earnings. However, the rate for individuals with significant cognitive disabilities in the supportedemployment cohort was not much less — at almost 95 percent — than for the other two subgroups. On the otherhand, the percentage of individuals reporting earnings in the sheltered employment cohort with severe cognitivedisabilities was only 54%. This rate is somewhat higher for the cohort of individuals with mild cognitive disabilities insheltered employment, at roughly two-thirds. Finally, 74% of the cohort with moderate cognitive disabilities in shel-tered employment reported earnings.
Pre- and Post-VR Longitudinal Earnings
The longitudinal earnings records were then aligned around the specific quarters during which application and closure from VR services took place. This process resulted in two time intervals – pre-enrollment and post-closure in vocational rehabilitation. The results of this procedure are presented in Table 7, which presents the earn-ings chronologies for the two primary cohorts. The second and third columns of the table report the percentage ofindividuals who potentially could have had VEC reported earnings in the given year that actually had VR earnings. Inthe three years prior to employment, 26% of the supported employment cohort had VEC reported earnings in thethird year prior to enrollment, 32% in the second year prior to enrollment, and 41% in the year immediately precedingenrollment. In other words, the supported employment group was less likely to have reported earnings as the lengthof time prior to enrollment increases. In contrast, the sheltered employment group was less likely to have reportedpre-employment earnings than the supported employment groups, and these earnings were much more stable overtime, ranging from 13%-16%.
Table 6 -- Percentage of Sample Members with Earnings Reported to
the Virginia Employment Commission, Sorted by Diagnostic Level
Supported Employment
Sheltered Employment
Employment
Moderate
Moderate
The VEC maintains a file of quarterly earnings records for persons working in “covered” employment. Cov- ered employment refers to employment situations in which the person is eligible to receive unemployment insurancebenefits should the need arise. Historically, about 80% of individuals receiving VR services in Virginia have beentracked by the VEC in at least one quarter when tracked over a ten-year interval. Examples of groups not covered bythe VEC include workers employment outside state boundaries, workers in organizations with less than four employ-ees, along with workers employed in Federal agencies, religious organizations, and certain non-profit organizations.
In particular, Section 60.2-213 B3 of the Virginia Unemployment Compensation Act, 1998 exempts rehabilitationfacilities with sheltered employment from covered employment. As such, the earnings records reported below do notinclude individuals who may have had earnings resulting from sheltered employment that were not reported to VEC.
Table 7 on the following page also provides annual earnings totals for both cohorts prior to and after partici- pation in the vocational rehabilitation program. The figures in Table 7 represent the average dollar value of earnings,in the four-quarter period, for those individuals with any VEC reported earnings during the calendar year (i.e. a “non-zero” mean). For individuals in the supported employment cohort, pre-enrollment earnings ranged from $2,100 to$2,300 in the three years prior to enrollment in vocational rehabilitation. In the years after program participation,earnings immediately increased by approximately 50%, and then stabilized in the range from $3,600 to $3,800annually. In contrast, earnings for members of the sheltered employment cohort range from $1,600 to $1,900 annuallyprior to enrollment. After program participation, the sheltered employment cohort’s earnings increased by approxi-mately 10%-20% and then stabilized between $2,000 and $2,500 per year. Mean cumulative earnings of the sup-ported employment cohort across the entire seven-year post-closure were $18,945, compared to $8,364 for thesheltered employment cohort.
Table 7 -- Earnings Chronology -- Pre and Post VR Program
Participation - Total Sample
Percentage Reporting Earnings
Mean Earnings
Supported
Sheltered
Supported
Sheltered
Table 8 on the following page presents the mean annual earnings for the supported and sheltered cohorts, sorted by the three classification levels. In most cases the pre-enrollment VR earnings levels decline as expectedacross the three class classifications of mild, moderate, and severe cognitive disabilities. The pre-employmentearnings levels are, on the whole, lower for the three sheltered cohorts when contrasted with their counterpartsreceiving supported employment.
The post-program earnings data contained in Table 8 vary significantly between the sheltered and supported employment cohorts. The amount of annual earnings varies little across the three supported employment cohortsduring this interval. Not surprisingly, persons with mild cognitive disabilities have the highest VEC reported earningsin every period, followed by the individuals with moderate cognitive disabilities and persons with severe cognitivedisabilities. The total earnings over seven years for these subgroups range from $21,000 to $19,000 to $15,000.
A much different story emerges for the sheltered employment cohorts. The level of earnings is much lower than for the supported employment cohorts. There is also a relatively wider variation across the three disabilityclassifications, with earnings again diminishing across the subgroups of individuals with mild, moderate and severecognitive disabilities. Mean annual earnings for persons with mild cognitive disabilities range from $2,300 to $3,300over the seven-year span, with no readily apparent pattern. Total earnings over the entire post-program intervalaverage about $11,000 and are some $10,000 less than their counterparts in supported employment. The annualearnings for individuals with moderate cognitive disabilities are lower, ranging from $1,700 to $2,700, again with noapparent trend. The total earnings stream of roughly $9,000 over the seven-year interval is more than $10,000 lessthan the earnings for the comparable cohort in supported employment. Finally, individuals with severe cognitivedisabilities had earnings ranging from $600 to $1,900, with an increasing trend over most of the seven-year period.
The total earnings over the seven-year interval average only $5,000, almost $10,000 less than their counterparts insupported employment.
Table 8 -- Mean Earnings - Pre and Post VR Program Participation
Sorted by Diagnostic Level
Mile Cognitive
Moderate Cognitive
Severe Cognitive
Disability
Disability
Disability
Supported
Sheltered
Supported Sheltered Supported
Sheltered
Testing for the Validity of Comparing Pre-Program Earnings for the
Supported and Sheltered Employment Cohorts
The previous section reported the pre-and post VR earnings experiences of the sheltered and supported employment cohorts. The following sections develop and implement a framework for making direct comparisons ofearnings outcomes between persons receiving sheltered vs. supported employment. A complete description of theprocedures used to conduct the comparison analyses are reported elsewhere (Dean, Dolan & Schmidt, 2001) The methodology employed is based on the usage of a “comparison group”. In the typical application, a comparison group is chosen that is similar in all respects to the “treatment” group, with the exception that the former isnot exposed to the treatment regimen. Tests are then undertaken to detect for the presence of “selection bias”. Theconcern is that any treatment effects on post-program earnings may be contaminated by an unmeasured factor (suchas motivation or level of functioning) that impacts on both the person’s decision to participate in a job-training programand their subsequent vocational outcomes. Biased treatment effects on earnings result if selection bias is presentand not sufficiently “controlled for” through the appropriate econometric correction.
In this situation involving vocational outcomes of persons receiving sheltered or supported employment, both have applied and been accepted for vocational rehabilitation and then exposed to some form of vocationalservices. However, there may be some programmatic screening by vocational rehabilitation counselors or otherservice providers that filters persons with certain measured and/or unmeasured traits into one program or the other.
Thus, tests for selection bias can be used to determine if both groups are drawn from the same population in thatthey do not differ in terms of such observable and unobservable factors that influence their pre-program earnings. Ifthe statistical tests involving the two groups are “passed”, it is then defensible to compare their post-program earn-ings.
A series of “Hausman Tests” were performed to test the significance of a treatment binary on differences in pre-program earnings. The Hausman tests on pre-program earnings changes reveal almost no unobserved differ-ences between the two groups participating in sheltered or supported employment. “Chow tests” were then used todetermine whether the effect of pre-program variables (e.g. diagnostic level, education, etc.) is the same for both thesheltered and supported employment groups. The Chow tests for the third vs. second pre-program year are insignifi-cant in five of the six gender/diagnostic label classifications, with only men with mild cognitive disabilities revealingstatistically significant differences in the factors influencing pre-program earnings. The Chow tests for the second vs.
first pre-program years were universally insignificant. Together, the Hausman and Chow results suggest that thesheltered employment cohort serves as a valid comparison group for the supported employment group within theFixed Effects Modeling (FEM) analyses. A complete report of all comparison group tests is found in Dean, Dolan,and Schmidt (2001).
The FEM procedure is applied to a panel of five post-VR years for each individual. Consequently, there will be five separate treatment coefficients for each gender and diagnostic level that trace out a time path of the dollarvalue of the net impact of supported employment in comparison to sheltered employment. The Chow tests per-formed previously precluded the possibility of pooling the all persons with cognitive disabilities into a single earn-ings regression (i.e., not controlling for diagnostic classification). Additional tests were conducted to determinewhether this single pooled regression, but now with separate binary designations for mild and moderate cognitivedisabilities (using the subgroup of persons with severe cognitive disabilities as the reference group), adequatelycontrolled for the impact of the severity of the disabling condition on earnings. Once again, Chow test results indi-cated that there were statistically significant differences in the influences of the various explanatory variables on post-intervention earnings depending on the severity of the mental retardation. In essence, the results of these testsreveal that pooling the cohorts of persons with cognitive disabilities with varying diagnostic levels in a single regres-sion is not warranted. Accordingly, the correct specification of FEM earnings estimates stratifies the regressionequations by gender and diagnostic level of cognitive disabilities.
There are six FEM estimates of supported employment earnings impacts presented in Table 9 on the following page, with separate reporting for women and men with either mild, moderate, or severe cognitive disabili-ties. The five rows contain the panel estimates for each of five post-program closure years. The reported values ineach cell measure the dollar amount of earnings differences between cohorts receiving supported versus shelteredemployment in a given year. T-ratios, with asterisks designating .05 and .01 levels of significance are reported inparentheses below each earnings coefficient. Thus, the “1824.11” figure in the first cell infers that women with mildcognitive disabilities in supported employment services had a statistically significant (at the .01 level) $1,824 more inVEC-reported earnings than their counterparts in sheltered employment, after controlling for both varying and constantindividual-specific and economy-wide factors. Moreover, the treatment effects in the first year after closure for bothwomen and men (i.e., the results reported in the entire “1st” row) are positive and significant across all three diagnos-tic levels and range from $1,437 to $1,870.
The sustainability of these earnings gains for recipients of supported employment over the five years after closure from the program differs across cohorts. Earnings gains decline, and eventually become statistically insig-nificant, for women with either mild or moderate cognitive disabilities. On the other hand, earnings gains increaseover time for women with severe cognitive disabilities. A different picture emerges for the men. Males with mild cognitive disabilities experience stable earnings gains, while those with moderate and severe cognitive disabilitiessee declines in the earnings gains over the five years, although the gains are still statistically significant.
Table 9 -- Fixed Effects Modeling (FEM) Earnings Equations by
Diagnostic Level Across Five Years of Post-Program Earnings
Year After
Moderate
Moderate
* Significant at .05 ** Significant at .01 Discussion
The present study investigated the long-term earnings impact of sheltered and supported employment on two cohorts of individuals with cognitive disabilities served by a state VR agency. Data on demographic characteris-tics, pre- and post-program earnings, and local economic conditions were merged to allow a comprehensiveexamination of the earnings outcomes of individuals who receive various types of VR services. First, the cohorts ofindividuals in sheltered and supported employment were compared to other individuals with cognitive disabilitiesidentified by the state VR agency as successfully rehabilitated. Second, the demographic characteristics and pre-program earnings of the two cohorts were compared to identify differences between the individuals served in the twoprograms. Third, post-program earnings histories were prepared and examined to discover trends in the post-program experiences of the two cohorts. Finally, a FEM analysis was used to examine the effect of the two serviceson post-program earnings while controlling for the influence of demographic factors, pre-program earnings, andchanges in economic conditions. Several key findings emerged.
Persons served in sheltered and supported employment programs differ significantly from other indi-
viduals successfully served by the state VR agency. Members of the sheltered and supported employment cohorts
were nearly twice as likely to be classified with a moderate or severe cognitive disability and five times more likely
to be an SSA beneficiary than a comparison group of individuals with cognitive disabilities closed as successfully
rehabilitated into Status 26. In addition, the sheltered and supported employment cohorts were more likely to be
female and less likely to be African-American than the Status 26 group.
Individuals in supported employment were twice as likely their counterparts in sheltered employment to
have worked in competitive, integrated employment settings prior to program entry. Three years prior to program
entry, 26.9% of supported employment participants engaged in VEC covered employment. This employment rate
rose to 41.4% in the year immediately prior to program entry. In contrast, the rate of participation in VEC covered
employment for persons in the sheltered employment cohort remained quite stable, ranging from 13.7% to 16.4% in
the three years prior to program entry.
As noted previously, individuals in sheltered workshops would generally not have their workshop earnings reported to VEC. It is highly likely that a large percentage of individuals in both cohorts not engaged in VEC coveredemployment were actually in some type of segregated employment setting at the time of program entry. It appearsthat the key difference between the two cohorts is that a larger percentage of the supported employment cohort wasengaged in some type of competitive, integrated employment in the three years prior to program enrollment.
It is important to note that when annual earnings of the individuals in VEC covered employment were com- pared across the two cohorts, no statistically significant differences were found. It can be reasonably assumed thatindividuals in the two cohorts were working about the same number of hours per week in generally the same types ofemployment. The difference between the cohorts appears to rest in their engagement with the competitive laborforce prior to program entry.
The comparison of the demographic characteristics of the sheltered and supported employment cohorts
revealed subtle differences. When the entire sheltered and supported employment cohorts are directly compared,
few differences are found. However, when subgroups of persons with mild, moderate and severe cognitive disabili-
ties are examined, a number of significant differences are revealed.
Persons with moderate cognitive disabilities comprise the majority of both groups,although the supported employment group had a higher percentage of individuals withmild cognitive disabilities and a lower percentage of individuals with severe cognitivedisabilities.
In the overall cohorts, no gender differences were found, although persons with mildcognitive disabilities in sheltered employment were more likely to be female than theircounterparts in supported employment.
No racial differences were found when subgroups with mild or moderate cognitivedisabilities were compared. However, African-Americans were more heavily repre-sented in the supported employment subgroup with severe cognitive disabilities thanin the sheltered employment group.
Persons in sheltered employment are generally more likely to receive SupplementalSecurity Income (SSI). However, in the subgroup of persons with severe cognitivedisabilities, SSI receipt was actually greater among persons in supported employment.
The two groups were equally likely to Social Security Disability Insurance (SSDI)benefits.
Earnings of the supported employment cohort were 250% greater than the sheltered employment cohort
across the seven-year post-program period. In the years after program participation, earnings for supported
employment participants increase approximately 50% and then become stable. For people in sheltered
employment, earnings increase from 10%-20% immediately after program participation and stabilize as well. Similarpatterns were found across all subgroups of persons with mild, moderate and severe cognitive disabilities.
Results of the fixed effects modeling (FEM) procedure, which controls for differences in demographic
characteristics, pre-program earnings, and local economic conditions, indicate that supported employment had
a statistically greater impact on earnings when compared to participation in sheltered employment. For males
with mild, moderate, or severe cognitive disabilities, and females with severe disabilities, these differences were
significant across all five of the post-program years reviewed. For females with mild cognitive disabilities, significant
differences were only found during the first two post-program years. For females with moderate cognitive disabilities,
significant differences were found in the first four program years, but not in the fifth year.
Limitations
The present study possesses a number of limitations. First, and most significantly, the study investigated the outcomes generated by one public program (vocational rehabilitation) in a single state. The demographic characteris-tics of the VR applicant population, the policies and operational guidelines in effect during the service period, thefluctuating economic conditions, the availability of both sheltered and supported employment provider agencies, andnumerous other factors unique to Virginia combined to influence the reported outcomes. As such, it is not possible togeneralize the results of the investigation to all public vocational rehabilitation programs nationwide.
At the same time, during the period of the investigation, Virginia operated a statewide network of sheltered and supported employment programs that were nationally recognized for their size and quality. The individualsserved in both programs differed substantially from other persons served through the vocational rehabilitation pro-gram. The availability of both sheltered and supported employment provider agencies in all areas of the stateinsured, to the extent possible, that decisions to serve individuals in a particular program were based on program-matic considerations, as opposed to the availability of only one type of program alternative in a specific community.
While generalization the results is clearly not appropriate, Virginia may be viewed as an illustration of a vocationalrehabilitation program providing “state-of-the-art” services in the late 1980s and early 1990s.
A second limitation relates to the extent to which the two groups were represented in the state Unemploy- ment Insurance (UI) database (VEC covered employment). Virtually all members of the supported employment group(95%) were represented in the VEC UI database. A sizable number of individuals in sheltered employment were notfound in the VEC data and were not included in the FEM analyses. This lack of representation was presumably dueto the individuals not working in covered employment. Whatever the reason, the difference in the degree to which thetwo groups were represented in the UI database affected the overall analyses. However, it should be pointed out thatif the reason an individual was not represented in the UI was because he or she had no earnings from competitive,integrated employment during the post closure period, excluding those individuals from the analyses served toincrease the average earnings of the sheltered employment group. In reality, had it been possible to obtain accurateearnings profiles for the sheltered employment sample members not included in the UI database, the difference inlong-term earnings between the two groups would have increased significantly.
A third limitation of the current study is that it describes the activities of a single vocational rehabilitation agency. Program applicants were not randomly assigned to the two service programs. Likewise, the sheltered andsupported employment groups were not matched on various demographic characteristics. However, the two groups were remarkably similar on a number of different demographic characteristics and together they differed significantlyfrom all other individuals with cognitive disabilities served by the agency during the same time period. The demo-graphic and functional characteristics of sheltered and supported employment participants in other states should becarefully investigated to determine the similarities and differences that exist between persons served in the twoprograms.
Implications for the Present Study on the Future Design of Employment
Programs for Individuals with Cognitive Disabilities
The combined results of the present investigation provide general support for the U.S. Department of Education’s decision to amend the definition of employment outcome in the Federal-State VR program. There is littleevidence to indicate that sheltered employment significantly enhanced the employment status or economic self-sufficiency of individuals with cognitive disabilities in the study sample. Less than one-third of the individuals placedinto extended employment by the state VR agency engaged in VEC covered employment at any time after programparticipation. Annual earnings for those individuals who did participate in the competitive workforce after placement insheltered employment averaged less than $200 per month.
In amending the definition of employment outcome, the Department of Education stressed that for some individuals, sheltered or extended employment could serve an important transitional role as an interim step in anindividual’s path toward competitive, integrated employment. In the present study, the lack of significant movementinto competitive employment on the part of the sheltered employment cohort gives only limited support to thisassertion. Participation in competitive employment increased by only 10%-15% above pre-program rates for thisgroup and earnings were quite low. While these gains should not be totally discounted, the present investigationdoes little to counter the claims of critics (Murphy & Rogan, 1995) who argue that sheltered employment programsgenerally fail to prepare individuals for future success in the competitive labor force.
The contrasting outcomes generated by sheltered and supported employment would be readily explained if the two programs served entirely different populations. In this study, a number of statistically significant differenceswere found between the two cohorts in comparisons of demographic characteristics and pre-program earnings. At thesame time, there appears to be tremendous overlap between the two groups. For example, the majority of bothgroups were comprised of individuals with diagnostic labels of moderate cognitive disabilities, no significant differ-ences were found in the pre-program earnings of individuals engaged in competitive, integrated employment, andmembers of the two groups were equally likely to be SSDI beneficiaries. When compared to other groups of indi-viduals served by the state VR agency, individuals in sheltered and supported employment seem to be more alikethan they are different. Future research efforts should further investigate the complex factors that determine whetheran individual with cognitive disabilities participates in sheltered or supported employment.
In this study, the employment outcomes generated by supported employment significantly exceeded those of sheltered employment. However, when the employment participation rates and annual earnings of supportedemployment participants are viewed in absolute terms, two important shortcomings are revealed. First, the total post-program earnings of the supported employment cohort, while over twice those of the sheltered employment group,are quite low. Second, the rate of employment participation of the supported employment group declines over theseven-year post-program period, as individuals, for whatever reason, are unable to maintain their jobs in the com-petitive workforce.
It is important to note that a sizable majority of both cohorts are individuals who receive either SSI or SSDI benefits. In light of the significant disincentives to employment that existed in the SSI and SSDI programs during theperiod of investigation, it seems likely that many individuals’ earnings were intentionally restricted, so as not tojeopardize their benefit status. Additional long-term investigations should be initiated to determine whether the largenumber of recent reforms to the SSA benefits programs will promote improved employment outcomes for individualsserved in supported employment programs.
Future research and evaluation activities should focus particular attention on the impact of VR service pro- grams on female clients. In the present study, females with mild cognitive disabilities were over represented shel-tered employment. In addition, the positive earnings impact of supported employment participation declined overtime for females with mild, as well as moderate, cognitive disabilities. Sufficient information is not available to explainthese important differences. However, the disproportionate impact of supported employment on females, if replicatedin subsequent evaluations, would be a matter of significant concern.
Conclusion
A significant amount of evidence indicates that integrated employment options improve consumer employment outcomes, cost less than other adult day programs, and generate savings for taxpayers. Benefit-cost analyses completedin the last ten years indicate that integrated employment options dramatically improve individuals’ earnings and economicself-sufficiency (Cimera, 1998; Kregel & Wehman, 1997). Integrated employment options cost less than sheltered work-shops, activity centers or other day support options for individuals with disabilities. Local employment programs can servemore individuals for the same amount of money, and achieve better outcomes, if they adopt an integrated employmentapproach. Integrated employment options also generate substantial savings for taxpayers. It has been repeated demon-strated that supported employment programs lead to a decrease in dependence on federal disability benefit programs, areduction in the need for costly alternatives such as workshops or activity centers, and an increase in the taxes paid byworkers with disabilities.
The present study added significantly to the body of evidences that documents the relative efficacy of supported employment in comparison to segregated, sheltered employment alternatives. The ten-year longitudinal analysis of 877individuals with cognitive disabilities found that (1) supported employment options resulted in significant and sustainedimprovements in consumers employment and earnings outcomes, as compared to sheltered employment alternatives, and(2) surprisingly little difference exists in the demographic profiles of persons served in integrated versus sheltered employ-ment settings. In other words, the decision to serve individuals in integrated or sheltered programs is often based onprogrammatic or funding concerns as opposed to the functional characteristics of the individual, often to the detriment ofpersons with cognitive disabilities.
This investigation attempted to contribute to the ongoing discussion of the appropriateness of the Department of Education’s decision to modify the definition of employment outcome, as well as add to the body of research thatcompares the outcomes generated by sheltered and supported employment. The study is unique in a number ofways. First, it sought to reduce “selection bias” by evaluating differences in the two groups under comparison. Futureinvestigations that promote the superiority of one program alternative over another should carefully and completelydescribe the characteristics of the populations being studied.
Second, the study used comprehensive analytical procedures that examined the impact of vocational rehabilitationservices on post-program earnings, while controlling for the influence of demographic characteristics, socio-eco-nomic factors and local economic conditions. While the fixed effects modeling procedure proved appropriate forindividuals with cognitive disabilities, its application to other disability populations requires further investigation.
Finally, the methodological framework employed in the study led to the construction and analysis of ten-year earnings profiles that allowed the comparison of both pre and post-program earnings. Investigations of the long-termeffectiveness of various employment alternatives should refrain from reporting “snapshot” information from a singlepoint in time. Post-program earnings should be compared to individuals’ pre-program earnings histories to moreaccurately measure the impact of a specific program. Perhaps most significantly, an individual’s annual or monthlytotal earnings should be the “unit of analysis” for program comparisons. Program outcome studies that report onlyhourly wages or total wages for a group of individuals fail to allow specific comparisons or account for an individual’sSSA benefit status on post-program earnings.
References
Bellamy, G.T., Rhodes, L., Bourbeau, P., & Mank, D. (1986). Mental retardation services in sheltered workshops and day activity programs: Consumer benefits and policy alternatives. In F. Rusch (Ed.), Competitive employment issues andstrategies (pp. 257-271). Baltimore, MD: Paul H. Brookes Publishing Company.
Cimera, R. (1998). Are individuals with severe mental retardation and multiple disabilities cost-effective to serve via supported employment?, Mental Retardation, 36(4), 280-292.
Coker, C., Osgood, K., & Clouse, K. (1995). A comparison of job satisfaction and economic benefits of four different employment models for persons with disabilities. Menomonie, WI: Rehabilitation Research and Training Center on ImprovingCommunity-Based Rehabilitation Programs, University of Wisconsin – Stout.
Dean, D., Dolan, R., & Schmidt, R. (2001). Use of fixed effects modeling to assess the earnings impact of sheltered and supported employment on vocational rehabilitation clients with cognitive disabilities: Comparison group testing and fixed effectsresults. Richmond: Bureau of Disability Economic Research, University of Richmond.
Dean, D., Dolan, R., Schmidt, R., Wehman, P., Kregel, J., & Revell, G. (2001). A paradigm for evaluation of the Federal- State vocational rehabilitation program. Richmond, VA: Rehabilitation Research and Training Center on Workplace Supports,Virginia Commonwealth University.
Department of Education (Monday, January 22, 2001) Part XVI 34 CFR Part 361 State Vocational Rehabilitation Services Program; Final Rule VerDate 11< MAY> 2000 19: 17 Jan 20, 2001 . http://www.ed.gov/legislation/FedRegister/finrule/2001-1/012201a.pdf - 463900 bytes - Mon Jan 22 14:13:43 EST 2001. Kregel, J., & Wehman, P., (1989). Supported employment: Promises deferred for persons with severe handicaps.
Journal of the Association for Persons with Severe Handicaps, 14(4), 293-303.
Kregel, J. & Wehman, P. (1997). Supported Employment: A decade of employment outcomes for individuals with significant disabilities. In W.E. Kiernan & R. L. Schalock (Eds.), Integrated employment: Current status and Future Directions(pp. 31-48). Washington, DC: American Association on Mental Retardation.
McLoughlin, C., Garner, J., & Callahan, M. (1987). Getting employed, staying employed. Baltimore: Paul H. Brookes Murphy, S., & Rogan, P. (1995). Closing the shop: Conversion from sheltered to integrated employment. Baltimore, MD: TenBroek, J., & Matson, F. (1959). Hope deferred: Public welfare and the blind. Berkeley: University of California Press.
Wehman, P. (1981). Competitive employment: New horizons for severely disabled individuals. Baltimore: Paul H.
Whitehead, C. (1979). Sheltered workshops in the decade ahead: Work, wages, or welfare. In G. Bellamy, G.
O’Connor, & O. Karan (Eds.), Vocational rehabilitation of severely handicapped individuals. Baltimore: University Park Press.
Wehman & Kregel (1995). Supported employment: at the crossroads. Journal of the Association for Persons with

Source: http://www.szentlazar.hu/media/pdf/2010/04/sheltered-vs-supported-article.pdf