SEELS must meet the information needs of a wide variety of audiences using a variety of data collection and analytic approaches. Accordingly, the SEELS sample must meet the following requirements in order to serve its multiple purposes:

• Focus on students. SEELS data must produce accurate estimates about the characteristics, programs, and outcomes of students in special education. However, no list of all students in special education exists from which to draw the SEELS sample. Thus, a sample of LEAs must be drawn, from whose rosters students can be selected. However, the sample of LEAs is only a vehicle to obtaining a sample of students; it is too small to make highly precise national estimates about LEA practices (OSEP is commissioning a separate study of state and local implementation of IDEA97 to meet this latter purpose).
• Generalize to each disability category and age cohort. Not only must the SEELS sample enable reasonably precise estimates for the full special education student population ages 8 through 12 at the outset, OSEP requires that it also generalize to each special education disability category and to each of the single-year age cohorts within the age range. This requirement has important implications for the size of the student sample, which must have enough students in each disability category to meet this requirement. If the sample contains sufficient numbers of students per category, it also will be large enough to generalize to the five single-year age cohorts within the sample.
• Longitudinal. SEELS data will be collected repeatedly over a 5-year period (see the timeline presented previously). The initial sample must be large enough to support estimates of reasonable precision in the fifth year of data collection (assuming that 8% of students who are in the sample each year will be lost the following year because of mobility).
• Multiple data sources. As the data collection plan outlined, multiple data sources will be needed to obtain the breadth of information specified in the SEELS conceptual framework. Many analyses will employ information from more than one source. Given reasonable assumptions about response rates to the various data collection efforts, some students will not have information from a source, reducing the sample for analyses using that data source. Even more will be missing information when several sources are combined. The sample must be large enough to accommodate missing information from multiple data sources.
• Multiple analytic purposes. The richness of the SEELS database will support a variety of analyses that have implications for the sample design. For example, subgroup analyses will examine experiences and outcomes of students in special education who are differentiated by particular characteristics (other than age and disability category, as mentioned above), such as gender, ethnicity, or functional abilities. The SEELS sample must be large enough to support such subgroup analyses.

In the remainder of this section, the approach to meeting these sample requirements is presented.

SEELS will employ a two-stage process to generate the needed sample of students in special education between the ages of 8 and 12. SEELS will draw a random sample of students in special education from a nationally representative sample of LEAs and a sample of state-supported special schools. Accordingly, the LEA is the primary sampling unit and the student with a disability is the secondary or final unit.

The SEELS sample will be generated by randomly selecting special education students from rosters of LEAs and state-supported schools that serve children of the appropriate ages in special education. The universe of eligible LEAs and special schools will be stratified by key factors to enhance representativeness; these factors are geographic region, district enrollment, and district/community wealth. Taking into account the length of the data collection period and assumptions regarding attrition from the sample, analyses of statistical power requirements suggest that an initial sample of approximately 12,075 students will yield a sample of sufficient size and representativeness to meet the analytic needs of SEELS in its final wave of data collection. This sample will be selected so as to generate 1,150 in each disability category, with the exception of 200 students who are deaf-blind and 375 with traumatic brain injuries, the least populous categories.

The following sections describe the process through which the student sample size was determined and then outline the selection procedures for the LEA and student samples.

The size of the SEELS student sample is a function of the duration of the study, desired levels of precision, and assumptions regarding attrition and response rates. The following assumptions have been used in determining the size of the student sample:

• Location information (parent name, address, telephone number) will be provided by LEAs for 90% of sampled students. Therefore, for each 1,000 students sampled in
  year 1, location information will be available for 900 students.
• In each year of the study after the first year, 92% of the students from the preceding year will be retained. Thus, for each 1,000 students sampled in year 1, 900 will have location information and the sample can be expected to retain 828 in year 2, 762 in year 3, 701 in year 4, and 645 in year 5.
• For each 1,000 students sampled in year 1, parent/guardian interviews will be completed for 70% of students retained in the sample, or 630 students in year 1
  (i.e. 70% of the 900 students whom we can track), 533 in year 3, and 451 in year 5.
• Direct assessments will be completed for 75% of the students retained in the sample each year. Therefore, for each 1,000 students sampled in year 1, assessments will be completed for 621 students in year 2 (i.e., 75% of 828 tracked students), 572 in year 3, and 484 in year 5.
• The number of students who have both a parent/guardian interview in year 1 and a direct assessment in year 2 will be 75% x 92% of the number of students who have a parent/guardian interview in year 1 (i.e., 435 students for each 1,000 sampled in year 1).
• The number of students who have both a parent/guardian interview and direct assessment in year 3 will be 75% of the number of students who have a parent/guardian interview in year 3 (i.e., 400 students for each 1,000 students sampled in year 1).
• The number of students who have both a parent/guardian interview and direct assessment in year 5 will be 75% of the number of students who have a parent/guardian interview in year 5 (i.e., 338 students for each 1,000 students sampled in year 1).

This means that approximately three students (i.e., 2.96 students) will need to be sampled for each one student who will have both a parent/guardian interview and a direct assessment in year 5 of SEELS.

The SEELS sample design emphasizes the need to estimate proportions and ratios (for example, the percentage of students in special education reading at grade level) instead of estimating the actual numbers of students in special education having specified characteristics (for example, 2,400 students reading at a particular level). However, relatively precise national estimates of the proportions or ratios of students in special education, whether analyzed as one group or considered separately by disability category, will be needed to adequately answer research questions of interest to the broad range of likely audiences for the study.

Excluding the deaf-blind, the average number of parent/guardian interviews in the NLTS was approximately 660 per disability category, resulting in a sampling efficiency of approximately 50% and standard errors for proportions of about 2.8%. This level of precision was used as a starting point for precision estimates for SEELS. To obtain the same level of precision in analyses that depend on students who have both the parent interview and the direct assessment in year 5, 1,954 students (i.e., 660 x 2.96) would need to be sampled per disability category.

This sample size is so large that it would make the study prohibitively expensive, particularly given the central importance and considerable expense of a direct assessment of students. In addition, a sample of that size would be a sizable proportion of all the students there are in some low-incidence disability categories (see Table 4-1). Students in special education account for approximately 10.6% of all students in American schools, and the number of students ages 8 through 12 in each disability category ranges from a high of approximately 1,111,000 for students with learning disabilities (approximately 4% of the elementary/middle school student population) to a low of approximately 560 for deaf-blind students (far less than 1%). The 1,954 students per category that would be needed to reach a precision level of 2.8%, as in the NLTS, would be 43% of all students in this age group with traumatic brain injury (TBI) and 16% of all students in this age group with orthopedic impairments. The SEELS sample would need to be drawn from an extremely large number of LEAs to find 1,954 students in these categories.

Given these drawbacks to sampling sufficient numbers of students per category to reach the NLTS precision levels, the following alternative strategy was used:

• Decrease the level of precision for standard errors to 3.6%. That is, by sampling 1,150 students per disability category (except for TBI and deaf-blind) in year 1, there would be 388 students per category with both a parent interview and a direct assessment in year 5. Assuming a 50% sampling efficiency (which will tend to be exceeded for almost all disability categories), the 388 students would achieve a standard error of estimate of 3.6%.
• Sample students with traumatic brain injury and students with deaf-blindness with certainty (i.e., select all students in those categories in sampled LEAs and special schools) but do not build the sample around those groups (i.e., do not select additional LEAs, beyond those needed to meet the sample requirements for other categories). The sample of LEAs and state-sponsored schools that is sufficient to meet the sampling requirements of other categories is estimated to contain approximately 375 TBI and 200 deaf-blind students, who will be sampled with certainty. This approach results in sampling far fewer LEAs to generate the sample and increases the sampling efficiency for the other disability groups.

The first step in developing a sample that leads to national estimates about students in special education is to select an adequate, representative sample of LEAs. Below we discuss issues related to the LEA sample including size, stratification, and fit.

There are several factors to consider in determining the number of LEAs for the sample. First, it is necessary to establish the number of LEAs that are required to generate the needed student sample. On the basis of an analysis of LEAs’ estimated enrollment across district size, and estimated sampling fractions for each disability category, 297 LEAs (and as many state-sponsored special schools as will participate) will be sufficient to generate the student sample. Second, the rate of LEA refusal to participate should be considered so that the required number of LEAs agree to participate within the limited recruitment period. Previous experience with the NLTS suggests that LEAs typically declined to participate because of concerns related to confidentiality of student records. Although considerable time and effort was expended in recruiting LEAs for the NLTS, approximately 55% of the LEAs invited to participate either declined, did not respond, or introduced procedures that unacceptably lengthened the recruitment process. In SEELS, both the amount of time and the funds available to recruit LEAs are less than were available in the NLTS. Efficiency can be gained if recruitment efforts focus on large LEAs, which are relatively few in number and from which a relatively large proportion of sample students will be selected. Smaller LEAs will receive less intensive recruitment effort than in the NLTS because there are many of them, yielding a large number of potential replacements for refusing districts. Although this strategy is likely to be most efficient in selecting the LEA sample quickly, there is a risk that smaller LEAs who refuse to participate differ systematically from other LEAs in terms of the types or effectiveness of programs that they offer to students. Thus, detailed tracking will be necessary to identify potential patterns that emerge with regard to LEA refusal/nonresponse. The procedural outcome of concentrating our recruitment effort on larger LEAs and being more willing to replace smaller LEAs is that a sample of 765 LEAs is expected to be enough from which to recruit 297 participating LEAs.

The initial task in selecting the SEELS sample is to define which districts should be included in and excluded from the universe of LEAs from which the sample will be selected. To meet its purposes, the SEELS sample includes only LEAs that have teachers, students, administrators and operating schools–that is, "operating LEAs." The SEELS sample excludes the following categories of local and state educational "districts" that appear on standard listings of educational institutions:

• Nonoperating LEAs, which do not administer any schools.
• Vocational-technical districts (except those that operate as regular LEAs). These districts generally do not serve students of the SEELS age range and often are not comparable to LEAs in enrollment, operating hours, or administrative structure, making their inclusion problematic.
• Supervisory unions, area educational agencies, interim districts, boards of county education services, or other superordinate units. These organizations occur most frequently in rural areas where the individual district-level enrollments are quite small. There is evidence that the operation of superordinate units varies from state to state. For example, in many states, the local districts are fiscally responsible for students attending such service units, and they are therefore listed on local district rosters. Such students would be included in SEELS. In other states, such units may have fiscal responsibility for their students. However, we believe that the exclusion of such units is justified for the following reasons: (1) their inclusion introduces the risk of double counting in states where students are found on local rosters and on superordinate rosters; (2) the numbers of students in such units are comparatively small enough that they could not serve as an analytic category.
• Public agencies, such as state education agencies (with the exception of the Department of Education in Hawaii, which is an LEA); Bureau of Indian Affairs (BIA) schools; achievement centers and regional resource centers; private agencies, such as homes for delinquent students; and Texas Independent State School Districts, which primarily are correctional facilities and homes for delinquent students. Included, however, are the "accommodation" school districts in Arizona, which are regular operating LEAs with nontraditional boundaries (e.g., around federal dams and military installations).·
• LEAs from Puerto Rico, Guam, and other territories, to reduce the cost and complexity of future data collection.
• LEAs that do not serve students in grades 2 through 7, which are most likely to encompass the age range of SEELS students (i.e., to be part of the sampling universe, an LEA must offer instruction in at least one of grades 2 through 7).
• LEAs (most with very small enrollments) for which the stratifying variable of district wealth cannot be obtained (see "Stratification" section for a discussion of the stratification variables).
• LEAs with 10 or fewer students in grades 2 through 7. Such schools would have an estimated enrollment of less than one student in special education in the target age range.

To create a sampling frame or master list of LEAs, two lists were considered: the public school universe maintained by Quality Education Data (QED, 1998) and the School District Name and Address File maintained by the National Center for Education Statistics in the U.S. Department of Education (1997). The two lists were compared on variables indicating LEA name and location. There were a small number of discrepancies in LEA name and contact information, probably because the data were drawn from different school years. The list with the most current information, QED, was used to construct the sampling frame. As a commercial source, it must maintain accurate data, including addresses of special education coordinators in each district, for its clients. The QED data are from the 1997-98 school year, as updated during the fall of 1998. Using the QED data, the following procedures were used to create a master list of LEAs that were eligible for the SEELS sample:

• Obvious errors were corrected, such as blank or duplicate records, no names, spelling errors, invalid codes, and extreme outliers.
• Sometimes two or three LEAs had the same name. LEAs with the same names were checked to make certain they were actually distinct LEAs.
• All nonoperating LEAs, supervisory unions, vocational-technical districts, and relevant public agencies were eliminated (see previous discussion), as were all districts that did not serve any grade in the grade 2 through grade 7 range.

These procedures resulted in a master list of 13,426 LEAs that are expected to have at least one student in special education in the appropriate age range. These comprise the SEELS LEA sampling frame.

The SEELS LEA sample is stratified for four principal reasons: (1) to increase the precision of estimates by eliminating between-strata variance, (2) to ensure that low-frequency types of LEAs (e.g., large urban districts) are adequately represented in the sample, (3) to improve comparisons with the findings of other research, and (4) to make SEELS responsive to concerns voiced in policy debate (e.g., differential effects of federal policies in particular regions, LEAs of different sizes). The first of these reasons is especially important because of the great diversity in the universe of LEAs. Three stratifying variables are used–geographic region, district size (student enrollment), and a measure of district/community wealth. They were selected on the basis of conceptual soundness and the likelihood of providing a gain in precision over simple random sampling. These variables and their sources are described below.

Region. This variable captures essential political differences, as well as subtle differences in the organization of schools, the economic conditions under which they operate, and the character of public concerns. Regions differ, for example, in the changes in school enrollment over time. They also differ in terms of economic health, which is linked to resources the region can target to education and other needed services. For SEELS, the regional classification variable selected is used by the Department of Commerce, the Bureau of Economic Analysis, and the National Assessment of Educational Progress (see Table 4-3).

By assigning each LEA to a region based on its state, we obtain the allocation of LEAs and proportion of total estimated elementary/middle school student population in grades 2 through 7 to region indicated in Table 4-4.

District size (student enrollment). LEAs vary considerably by size, the most useful available measure of which is pupil enrollment. A host of organizational and contextual variables are associated with size that exert considerable potential influence over the operations and effects of special education and related programs. These include the extent of district administrative/supportive capacity, the degree of specialization in administrative structure, the nature of citizen and interest group activity in education, and the characteristics of relationships with state and federal governance systems.

In addition, total enrollment (and the previously described estimated elementary/middle school enrollment) serves as an initial proxy for the number of students in special education served by a district. The QED database provides enrollment data from which LEAs were sorted into four categories serving approximately equal numbers of students:

• Very large (estimated enrollment greater than 17,411 in grades 2 through 7). These are either districts in large urban centers or large county systems, which are typically organizationally complex and likely to be divided into subdistricts.
• Large (estimated enrollment from 4,707 to 17,411 in grades 2 through 7). These are districts set in small to medium-sized cities or large county systems. They are also organizationally complex, but these systems tend to be centralized.
• Medium (estimated enrollment from 1,548 to 4,706 in grades 2 through 7). These typically are suburban districts, large rural towns, and small county systems.
• Small (estimated enrollment between 10 and 1,547 in grades 2 through 7). The majority of districts in the country fall into this group. Most are small rural districts, which most likely receive little money for special education programs; the range of activities that these funds can be used for is likely to be extremely narrow.

The distribution of districts among these strata and proportion of students accounted for by each stratum are displayed in Table 4-5.

District/community wealth. LEAs differ greatly in the resources they have available and in the demands placed on those resources by low-income students whose needs put them at risk for a variety problems, including school failure. Policies and programs may differ in LEAs that face these differential demands of disadvantaged students. Also, prior research has demonstrated that high-poverty districts also have a high proportion of students in special education. As a measure of district wealth, the Orshansky index (the proportion of the student population living below the federal definition of poverty) is a well-accepted measure. The distribution of Orshansky index scores was organized into four categories of district/community wealth, each containing approximately 25% of the student population in grades 2 through 7:

• High (0% to 12% Orshansky)
• Medium (13% to 34% Orshansky).
• Low (35% to 45% Orshansky).
• Very low (over 45% Orshansky).

The distribution of districts among strata and proportion of students accounted for by each stratum are displayed in Table 4-6.

The three variables generate a 64-strata grid into which the entire universe can be fit. Table 4-7 shows the strata and the number of LEAs in each stratum. Table 4-8 shows the number of students in all LEAs in each stratum. The next stage in the SEELS sampling process was to select the appropriate LEAs from each stratum to yield a total sample of 765 LEAs. LEAs were selected from strata so as maximize the sampling efficiency and thereby maximize the effective sample sizes.

Our first step in assessing the effectiveness of the sampling process was to evaluate the degree to which the selected LEA sample was comparable to the universe from which it was drawn on variables used in the sampling process. Tables 4-9, 4-10, and 4-11 depict the characteristics of the LEA sample, in weighted and unweighted form, on the sampling variables of region, LEA size, and LEA wealth. Taken together, the tables illustrate that the weighted LEA sample closely resembles the LEA universe with respect to those variables.

In addition to ensuring that the LEA sample matched the universe of LEAs based on variables used in the sampling, it was important to ascertain whether this stratified random sampling scheme resulted in skewed distributions on relevant variables not included in the stratification scheme. Two variables from the QED database were chosen to compare the "fit" between the first-stage sample and the population: the district’s metropolitan status (Table 4-12), and the district’s proportion of minority students (Table 4-13). If comparisons between the universe of LEAs and the sample revealed a poor fit, either the sample would be reweighted or a new sample would need to be selected. However, Tables 4-12 and 4-13 reveal that the fit between the weighted LEA sample and the LEA universe is quite good.

Because LEAs have an unequal probability of being selected into the sample, depending on the stratum within which they fall, LEAs will need to be weighted by the inverse of the stratum sampling fraction to create population estimates. As discussed previously, approximately 1,150 students must be sampled in the higher-incidence disability categories, 375 students with traumatic brain injury, and 200 deaf-blind students to make national estimates with reasonable precision about students in each category and students in special education overall.

During the design task, SRI will contact LEAs and obtain their agreement to participate in the study. Subsequently, in the fall of the 1999-2000 school year, the study contractor will request from participating LEAs rosters of students in special education between the ages of 8 and 12. Requests for rosters will specify that they contain the names and addresses of students in special education under the jurisdiction of the LEA, the disability category of each student, and the students’ birthdates or ages. As mentioned previously, some LEAs can be expected to provide only identification numbers for students, along with the corresponding birthdates and disability categories. When students are sampled in these LEAs, identification numbers of selected students are provided to the LEA, along with materials to mail to their parents/guardians (without revealing their identity to the study contractor).

After estimating the number of students enrolled in special education at the appropriate grade levels, the fraction of students in each category at each age that must be selected randomly from each district to yield a sample of 12,075 students must be determined. These sampling fractions will be calculated to maximize the effective sample efficiency while obtaining the required absolute sample sizes. Final sampling fractions cannot be calculated until the composition of the sample of participating LEAs is known; however, initial estimates are presented in Table 4-14.

In addition, from the state-supported special schools, 100% of students who are deaf-blind, 50% of students with visual impairments, and 15% of those with hearing impairments are expected to be sampled.

Student sampling weights are the product of the LEA sampling weights and the inverse of the student sampling fraction. The student sampling weight is the number of students in the universe represented by an individual student in the sample. Estimated sampling fractions and weights are included in Table 4-15. In addition, from the state-supported special schools, we expect sampling weights of 3.8 for the deaf-blind, 7.6 for students with visual impairments, and 25.8 for students with hearing impairments.

Table 4-16 contains a description of the activities required to complete the selection of the student sample.

To minimize sample attrition over the years of data collection, the SEELS study contractor will need to use aggressive tracking mechanisms to maintain accurate and up-to-date contact information for sample members. To aid in this task, the parent questionnaire will include information that will facilitate tracking of parents/guardians, such as additional work and home telephone numbers for the respondents, location information for one or more friends or relatives who would know where the family had moved, and e-mail addresses.

BACK

NEXT