The Effects
of Academic Career
Magnet Education
on High Schools
and Their Graduates

MDS-779

Robert L. Crain
Anna Allen
Robert Thaler
Debora Sullivan
Gail L. Zellman
Judith Warren Little
Denise D. Quigley


Institute on Education and the Economy
Teachers College, Columbia University

National Center for Research in Vocational Education
University of California at Berkeley
2030 Addison Street, Suite 500
Berkeley, CA 94720-1674

Supported by
The Office of Vocational and Adult Education
U.S. Department of Education

February 1999

FUNDING INFORMATION

• ACKNOWLEDGMENTS
• EXECUTIVE SUMMARY
• CHAPTER 1: Introduction by Robert L. Crain
• CHAPTER 2: Career Magnet Graduation Rates by Robert L. Crain and Robert Thaler
• CHAPTER 3: The Academic Effects of Career Magnets by Robert Thaler and Robert L. Crain
• CHAPTER 4: The Design of Career Magnet Programs and Students' Experience of High School by Debora Sullivan and Judith Warren Little
• CHAPTER 5: Career Magnet Schools: Effects on Student Behavior and Perceived Parental Support:
  Part One by Gail L. Zellman and Denise D. Quigley
  Part Two by Robert L. Crain
• CHAPTER 6: Placing the School-to-Work Transition in the Context of Adolescent Development by Anna Allen
• CHAPTER 7: Conclusions by Robert L. Crain
• APPENDIX A: Additional Notes on the Methodology
• APPENDIX B: Using the Experimental Results to Estimate the Impact of Career Magnets on Students

• Related readings for this publication

Acknowledgments

      This report owes a large debt to many people and institutions. Our greatest debt is to the students and educators who we interviewed; without them, this report would not exist. We are also indebted to the local educational administrators, especially the research staff, for permitting us to do this study and providing a great deal of technical assistance. In particular, we would like to thank the principals who gave us permission to interview students and staff in their schools. Since the schools are anonymous in this report, we are unable to thank them by name. We also owe a great debt to four, very wise and hard working (but again anonymous) school professionals who helped us understand their school, helped us select samples of students, and helped us solve a thousand problems which would have crippled our fieldwork.

      We would like to thank our staff of interviewers without whom this study would never have been completed: Janice Kelly, Kim Alkins, Susan Wilcox, and Ray Westbrook. In addition, we would like to thank five other staff members of this project who contributed so much to its fulfillment: Amy Heebner, Will Jordan, Yiu-Pong Si, Vanessa Smith, and Barbara Tokarska.

      We must also thank our best critics: Thomas Bailey, Director of the Institute on Education and the Economy (where this project was based); Sue Berryman, former Director of the Institute on Education and the Economy; our former colleague at Teachers College, Michael Timpane; and from the Manpower Development Research Corporation, Edward Pauly, George Cave, and David Long.

      In addition, we wish to thank the staff at Teachers College, Columbia University--especially George Schuessler and Evo Antoniazzi of the Computer Center, and Lisa Rothman of the Institute on Education and the Economy.

      Finally, we want to thank Morton Inger, whose editing, writing, and organizational skills enabled us to get this report ready for publication.

Executive Summary

      This study has identified a group of career magnet high school programs that have had some success in educating low and moderate income minority and immigrant students. The study has also identified ways in which these programs have not succeeded. The programs used an academic curriculum accompanied by coursework (and sometimes internships) to prepare students for specific careers. These career magnet programs--located either in regular comprehensive high schools or combined with other magnet programs to fill up an entire building--usually have the same budget as the regular comprehensive schools.

      The programs we studied are in a large area that includes a low-income city and a ring of older suburbs. Six out of every seven students are African American or Hispanic; the remainder are white, Asian, or Native American. Many of our conclusions are based on a comparison of a large number of students who had been randomly assigned through a lottery admission process to career magnet programs and to comprehensive schools.

      This study shows important positive and negative effects: While graduates of the career magnets are more likely to succeed in work and college, the career magnets also have a high dropout rate.

      Two kinds of studies were done:

CHAPTER  1
Introduction

      The United States high school is trapped in a dilemma. It must now teach all students because it is now widely assumed that every student should graduate from high school. Those who do not are called "dropouts," a term invented in the 1960s when not finishing high school became a failure to achieve the new norm. Furthermore, over 60% of high school graduates are enrolling in college immediately after high school. Now that high schools are no longer serving only the academically talented students, or at least the ones capable of obtaining the "gentleman's C" easily, the high school must find some way to motivate and teach students who are less well-prepared academically so that they will both want to and be able to graduate. But the high school is now chasing a moving target, as academic standards are being pushed upwards by political pressures of various kinds. Finally, there is pressure to increase equality of educational opportunity, and pressure from low-income parents who are convinced that their children must attend college if they are to survive economically in the modern world--which means that their children will not be going to vocational schools.

      This is a study of a group of high schools and programs within high schools in which students choose an academic career focus in eighth grade and leave their neighborhood for a regional career magnet school (or a regional program located in a distant comprehensive high school building). The careers the students choose usually require at least a community college education. The study suggests that however unlikely the notion of creating a school that is both career oriented and college oriented might seem, this is probably the type of school wanted both by the nation's leaders and by parents and students as well. Careers (if they can be made to work) may prove to be an ideal strategy for urban high schools. On the one hand, an academic career focus makes good sense for students who are unsure they will be able to graduate from college and have little access to occupational information either at home or in their community that will help them find quality employment. On the other hand, most of these students are not willing to trade the chance to attend college for even a good entry-level job. A high school that promises both college and a career allows them to make a choice without foreclosing their options.

      There is good evidence here that giving the college-preparatory high school an academic career focus creates a meaningful high school curriculum and provides a great deal of support to adolescents in this period of their development. Adolescent support is very important, and is an area where the traditional high school curriculum fails badly.

      These academic career magnets did not arise from some central planning process. They were the result of a gradual evolution. The district, with many poor and minority students, many immigrants, and much white flight, had many schools in crisis. In several cases, high schools were deemed to be such failures that they were closed and reopened with a new student body and staff. Other schools were closed because of declining enrollments due to demographic change or a loss of interest in vocational programs. Frequently, the school was located in an area where there were not enough students to support a school, and the school had to become a magnet. But vocational schools were not very popular, and there were already enough high school programs serving gifted students. By a mixture of default and foresight, local design teams of community residents, political leaders, and educators, who were given considerable power in the decisionmaking process, arrived at the idea of creating career magnet schools that were college preparatory without being restricted to the most highly talented students, attracting their students by having a particular focus--almost always a career. This academic career magnet strategy--we call it "career magnet" for short--became a successful recruiting device, and as the schools became popular and garnered more than their share of the city's best students, the principals of the comprehensive high schools fought back, creating their own "programs-within-a-school" in their high schools. By the late 1980s, a third of all the area's students were being educated in these schools and programs.

      In this area, the academically oriented career magnet may have been as much a bottom-up innovation as one imposed from the top down. The model has proven its political acceptability, its ability to garner support from parents and students, and its ability to be mass produced. The opportunity to evaluate program outcomes, to see if the model prepares a more skilled labor force, creates a more college-ready pool of high school graduates, or lowers the dropout rate, grew out of a happy accident: These schools, searching for the fairest possible way to admit students while being pressured to provide equality of educational opportunity, chose to fill one-half of those seats by creating a lottery system that made it easy for students to apply to the academic career magnet schools.

The Programs and Our Studies

      Career magnets recruit students by offering to prepare them for college and simultaneously provide them with an introduction to a particular career. In business, a program may teach accounting, financial management, word processing, and other business skills on the assumption that students will attend college, furthering their business education there. A typical program operates as a school-within-a-school, providing a number of classes separate from the rest of the school, typically teaching ten to twelve classes over a four-year period that are specifically focused on a career. Many have instructors with experience in that career. They take students on field trips to firms, and operate extracurricular clubs to learn more about careers. About half of them bring in lecturers from industry, use businesspeople as one-on-one mentors, or hold workshops on how to prepare a résumé. At the same time, they provide students the opportunity for a college preparatory education, and attempt to send a large number of students to community college. For many students, especially inner-city minority students, adult success may seem unattainable without college. They may live in a section of the city where few adults hold good jobs, and the only good jobs they know are held by people with college degrees. Attending a traditional vocational education program that focuses on employment rather than college would seem to be giving up too much, even for those who appear to have little chance of succeeding in college.

      The career magnet programs seem firmly committed to college preparation. For example, one career magnet program arranged its senior year schedule so that all students in the program spend one afternoon a week at a nearby college taking a course that will count toward their high school diploma and later as college credit. The program director describes this college course as popular with students, adding that it makes clear that career preparation is not incompatible with attending college.

      This report will focus on several research studies:

• We used the school records data,the official records of 9,176 students who attended 59 career magnet programs, comparing the programs to each other to identify strategies that were especially effective or particularly problematic.
  
  
• We used a survey,a 2¹/₂ hour interview with 110 high school graduates who had either attended career magnet programs in four school buildings or were members of a control group who had applied to the same programs but were randomly rejected and graduated from a comprehensive school.
  
  
• We analyzed life history data,four additional hours of interviews conducted over two days with 30 of the 110 graduates in the survey.

The Sites

      The 59 programs are located in 31 different high schools. Eight of the high schools are devoted entirely to career magnet programs, while the remaining 23 are comprehensive high schools that have a career magnet program within the school. For the survey of graduates and the life history interviews, we chose students who had applied to programs in four schools that were completely dedicated to academic career magnet programs, comparing lottery winners who had attended and graduated from the career magnets to students who had applied to the same four schools but had lost the lottery and had attended and graduated from comprehensive high school programs. We chose the four career magnet schools because they seemed to represent fully developed models. All four schools were at least ten years old and they collectively covered a wide range of career interests. We will refer to them as the "Health Careers magnet," the "Business magnet," the "Business Communications magnet," and the "Engineering magnet." In all cases, we were studying buildings that were entirely devoted to career magnet programs, even though the career magnet school within a comprehensive school is the more common model. We did so because we thought that the dedicated career magnet school building provided a clearer example that would be easier to interpret than the mixed model; we also thought that the school building devoted entirely to career magnet programs would be more typical of what other school districts in the United States would choose to implement. In each of the programs, we selected pairs of students who were matched on ethnicity, sex, achievement test scores, and neighborhood. One student in each pair had entered the lottery, attended the career magnet program, and graduated; the other lost the lottery, did not attend the career magnet program, and graduated from a comprehensive high school. Since we are not studying the students who dropped out, we are using a model that allows us to detect whatever differences occur when the two types of schools succeed in graduating a student.

The Schools

      "Health Careers" is a career magnet with only two programs: A small program, medical science, which focuses on theoretical medicine, seems primarily intended for college-bound students who are likely to enter nursing or pre-med programs in four-year colleges. The rest of the school is used by a large health careers program that includes eight different components, each preparing students for qualification in a particular area: practical nursing, nursing assistants, dental assistants, dental laboratory work, medical laboratory work, medical accounting, medical office work, and medical secretarial work. The wide variety of options within the program allows the school to serve students who vary greatly in their ability to work in front-line hospital service. Those who cannot tolerate blood or who cannot function safely in a clinical setting are provided a variety of office alternatives. At the same time, the academically strongest nursing students may move into the medical science program.

      The "Business magnet" high school contains seven business-oriented programs. The most prestigious is its program in securities and finance, but equally important are programs in accounting, business, computer science, business law, marketing, and secretarial science, an information systems program, which prepares students to work in a variety of word-processing positions. The presence of seven related programs gives the school the opportunity to reduce its dropout rate by moving students among the programs if they lose interest or are unable to do the work in one area.

      The "Business Communications magnet" is in many ways a similar school, but because it lacks the high-prestige finance program, it probably does not attract as many students interested in attending Ivy League colleges. Its three programs are (1) computer programming/accounting, (2) marketing, and (3) secretarial studies and word processing careers. Its advanced accounting students can do coursework at a partner college.

      The "Engineering magnet" has four programs. Two of its programs are in aerospace technology and computer science. In contrast, its other two programs, one in law and the other in television and other communications, provide a liberal arts complement. The student body is economically heterogeneous but entirely African American. Two of the programs have partnerships with local colleges: (1) The aerospace technology program is designed for students going on to engineering school and provides considerable work in engineering design. A small coterie of students are provided the opportunity to learn to fly. The program also offers opportunities to learn about the operation of electronic equipment and the maintenance of airport facilities; and (2) the communications program's main focus is to provide students with the opportunity to operate a television production facility in the school. However, it does not have good connections to the local television industry since its students and graduates must compete against college graduates even for unpaid internships. The other two programs do not have college partnerships in place. Computer science, which is primarily mathematical in its orientation, provides more computer theory than practice. The law is a popular subject with high school students. The program has a variety of internships, and it holds a mock trial competition each year. As with many of the programs, the academic career focus is broad, including preparation for police work and other aspects of law enforcement as well as preparation for college pre-law programs.

      All the career magnet schools must cope with the extremely wide range of students that they have, and most do so by moving students between programs. Sometimes a single program will be stratified, with a small group of students in the most advanced courses and others in an easier curriculum.

      Most career magnets receive no additional funds from the school board (a few have federal or foundation grants) and must reallocate funds within their regular budget to pay for any special equipment the programs require. The only exception are "redesign" funds, which become available to a school after it is deemed to be such a failure that it is closed and reopened with a new name and a new staff. These schools receive a supplemental appropriation for the first five years of their new life. Some total academic career magnet schools began their life as "redesigned" schools, making them the only academic career magnet programs that received extra funding.

      These career magnets provide a good opportunity to isolate particular elements of school-to-work programs, since some of the programs emphasize computers more than others, stress more visits to firms, or use more teachers with backgrounds in industry. The district's management system for these academic career magnets is highly decentralized. As a result, each program is free to create its own particular theme and to decide for itself many of the details of its curriculum and structure.

      While our study is of schools that are focused by their definition, the fact that the focus is specifically career-oriented limits the extent to which this study can be considered an evaluation of schools with focus. Nevertheless, it will contribute to our understanding of those schools as well as to the discussion of strategies for school-to-work transition. Its data will also be useful to the policymakers concerned with "school to work" and especially with "school to work for the college bound" (Bailey & Merritt, 1997) and "academies" (Kemple & Rock, 1996; Stern, Raby, & Dayton, 1992). This study also contributes to the research on choice, since these schools were constructed as choice schools--being magnets first, with the choice of theme coming second. Indeed, some of the programs created in this effort have no academic career focus at all.

      Finally, these programs in some ways resemble charter schools. Although they do not have charters and are not officially deregulated, all the high schools in this area are surprisingly autonomous. Decisions about what kind of program to operate, what changes in program should be made, and what new programs should be added are made with very little control from any school board or higher government administration. Nowhere in local or state government is there any educational administrator specifically responsible for career magnet programs, and, indeed, the official descriptions of the schools in this area do not even recognize the career magnets as a distinct type of high school program. Further, records do not identify whether students are in a particular program but only the building that they are in, so that a student in a career magnet program in a comprehensive high school is not identifiable as such statistically. This means that these schools are quite free to do as they wish, with little opportunity for the school system to regulate them through controls or even evaluate them. The schools are bound by the standard regulations governing personnel, which sometimes cause serious problems when schools need to recruit faculty who have specialization in their particular careers. But the looseness of these high schools, carried almost to an extreme, may mean that these schools provide interesting lessons for persons concerned with how charter schools may work out.

The Selection Process

      When the district's first academic career magnets were designed, it was intended that they be like most magnet schools in America--selective. However, the school board and administration has probably been more sensitive to issues of race and class segregation than most school districts in the country. When critics made the school board aware of the conflict between the magnet school's goal of being selective and the social goal of furthering racial and economic integration, a compromise strategy was gradually worked out that noticeably reduced the segregative nature of the selection process. First, in what appears to have been an effort to encourage students to apply to magnet programs, every middle school student in the area was required to fill out an application for high school (even if they planned only to attend their neighborhood high school), using a form that made applying to a magnet school as easy as possible. Secondly, each academic career magnet program could admit only one-sixth of its students from those with above-grade-level reading scores in the city's student body and another one-sixth from those reading below grade level (the other two-thirds coming from those within one standard deviation of the area mean). Finally, they required that half of the students in each of those three reading groups be admitted by lottery. That was a political compromise, since the principals of the career magnet high schools wanted to select all their students and the critics wanted them all randomly assigned; 50% was an obvious compromise point.

      If students wished to be considered for career magnet programs, they listed up to eight in order of priority. If they were also interested in programs at the four most highly selective schools in the area, they checked a separate set of boxes on the questionnaire; this arrangement meant they did not risk wasting their first priority choice by betting on a long shot. Finally, if they wanted to attend their neighborhood comprehensive high school, they could simply check a box on their application. The application process was spread over the entire fall semester of the eighth grade, giving students a chance to change their minds and to discuss choices with their parents.

      In late January, a subcontractor selected enough students to fill half the seats in each career magnet program. They did so by assigning random numbers to each student's choices, but preceded each random number with a number that represented that student's priority of choice. Thus, each student's first choice began with the number one, their second choice with the number two, and so on. The students with the lowest random numbers choosing each program were automatically offered admission. A waiting list was also created to replace decliners. Separate lotteries were performed for students with high, average, and low reading test scores. High was defined as one standard deviation above the mean, low was defined as one standard deviation or more below the mean or having no test data. Of the students randomly selected for each program, 16% were from the high- and another 16% from the low-reading groups, with the remaining 68% from the average reading group. Students whose test scores fell in the top 2% of the test distribution were exempt from the lottery and given their first choice assignment whenever possible.

      The files of students who had not been selected by lottery, including those who had been wait listed for the program, were then sent to the school, which selected enough students to fill one-half of the seats in the entering class from the remaining students. Students admitted to a program on the basis of their test scores being in the top 2% were counted as part of the top 16% of those selected by the school.

      The studies reported on here are based on a cohort of 9,174 students who applied to 59 different academic career magnet programs. (Students are also admitted into these programs in the tenth grade if they attend a seventh- through ninth-grade junior high, but we did not study these students.) Of these students, 2,373, or 26%, were admitted by lottery to their first choice academic career magnet program; of these, 63% entered ninth grade in that program. Of those who lost the lottery, 18% were school-selected and attended their first choice academic career magnet program. The remaining lottery losers were either admitted to one of the elite public high schools, a career magnet program that was not their first choice, entered ninth grade in their neighborhood comprehensive high school, or withdrew from the public school system (see Table 1.1).

      Of the students applying to these programs, 61% were female and 39% male. This imbalance may be because this sample of career magnets often focuses on careers in health and business, which are attractive career areas for females. It may also be that females are
more future-oriented at age 14, or that boys want to attend their neighborhood high school while girls want to escape from their neighborhood at that age.

      Student ethnicity is coded by the school board into five categories. For the cohort reported on here, the ethnic breakdown is as follows: 47% African American, 27% Hispanic, 8% white, 5% Asian American, and ¹/₂% Native American. Information concerning ethnicity is voluntary, and 986 students (12.5%) were not ethnically classified.

      We used the school district's method of assigning students by lottery to career magnet schools as the basis for our study. It is not immediately obvious that the lottery admission used in the school district was indeed a randomized experiment, and, in fact, it differs in certain ways from a traditional experimental design. But it does meet the two necessary conditions: (1) that subjects be randomly assigned to different treatments and (2) that outcome measures be taken after they have received the treatment. Lottery assignment to oversubscribed programs meets the first condition because it guarantees that some students will be randomly admitted to a particular program while other students who also applied to that program are randomly rejected. In all of our research we were studying only students who were lottery admitted to career magnet schools and graduated from them, comparing them to lottery-losing applicants to the same schools who had graduated from comprehensive high schools.

      We refer to the study as an experiment-based study rather than a classical laboratory experiment because we could not include every student who participated in the lottery. We could not study the effects of the programs on the students who left the study, attending private school or leaving the school district after having participated in the lottery. We could not study the postgraduate performance of students who did not graduate from the schools. In addition, when we did use the lottery as an experiment, we had to allow for the fact that some students were "misassigned," some lottery losing students attended the career magnet program despite having lost the lottery, and some lottery winners chose to attend comprehensive high schools. These omissions mean that our study should be thought of as based on an experimental model but not a perfect experiment. We have been as rigorous as possible in retaining the power of the original random assignment. Since in its initial step the lottery is exactly the same as the first step of a randomized experiment, we have stayed as close as possible to that original population.

      We examined the data for the first two years of the lottery (students admitted in 1987 and 1988), and students entering the fourth year in Fall 1990. The 1987 data showed deviation from randomness: lottery winners and lottery losers differed in their middle school performance to a significant degree. These differences were small but with an overall sample size of over 47,000, small differences are nevertheless statistically significant. The 1988 lottery selection showed a much smaller bias, although still statistically significant, and the 1990 lottery process shows no statistically different results at all. We spent the first year and to some extent the second year debugging the system until nonrandomness was eliminated. In our study, we used the data for the 1988 entering class of ninth graders.

      For each of the three reading groups and for every program (there were 136 programs in 1988) we identified the number of lottery winners and lottery losers and counted the number of students who actually entered the program after being admitted randomly. Since the lottery selection used the lowest random numbers, every student who applied to a particular program as his or her first choice had priority over students who applied to that same program as their second or higher choice. We then eliminated every program with fewer than nine students admitted to the program randomly and fewer than nine rejected or placed on the waiting list. We also eliminated programs where fewer than 60% of those randomly admitted actually attended the school, and those where more than 40% of the lottery rejected were selected by the school for admission after being rejected by the lottery. We thought the data from these programs would be meaningless because of so many assignment errors. In a few cases, we found that the entire set of first priority students were admitted because there were fewer students choosing the school as their first priority than there were seats available for lottery admission; in these cases, we looked to the students who had selected the school as their second priority choice to see if an experiment could be constructed using only the second priority choice students. We assumed that students who selected a school as their first priority are different from those who select a school as their second priority, so that simply comparing all lottery winners to all lottery losers would bias our sample, with the lottery winners being more likely to be first choice and the lottery losers to be later choices. For this reason, we made all our experiments within a single priority choice.

      When the selection of all experiments was completed we found that we had identified 112 experiments in 59 different programs involving 9,174 students. Some of the 59 programs were so large that we had valid experiments for all three reading levels, but more often a program provided valid data for experiments at only one or two reading levels.

      The first studies that we conducted were analyses of the effects of being randomly admitted to a career magnet program on the academic performance of students--their official school records of test scores, absenteeism, graduation, and dropping out. We also identified those characteristics of career magnet programs that had the largest impact on the students--either by increasing or decreasing the performance of the career magnet students compared to the lottery losers who had applied to the same program. For example, we found that when students applied to a program that provided students with more opportunities for hands-on computer work, lottery winners had higher math test scores than lottery losers, implying that the computer time was the factor that improved students' scores. We also used this sample to study graduation rates.

Methodology for the Survey and Life History Studies

      In the studies reported here, we turn from the general academic data on students to personal interview data. We conducted a set of interviews with a subsample of 110 graduates of career magnets and regular comprehensive high schools. In addition, 13 matched pairs of this subsample were asked back to give us life histories and to let us observe them as they worked together on a team work project.

      We concentrated on four schools, each made up entirely of career magnet programs. We did this because we expected schools entirely dedicated to career magnet programs would have more administrative support because they did not have to compete with a large comprehensive program, and, thus, would be able to focus on making their programs effective. We wanted to compare students who were admitted by lottery to the four schools, and who had subsequently graduated from them, to students who had lost the lottery, attended a comprehensive high school, and had graduated from there.

      We drew a random sample of the lottery winners and losers and deleted everyone who had not graduated from the high school within five years. To make the two groups as similar as possible, we selected graduates matched on the program they applied to, their home neighborhood, their test scores, ethnicity, and gender.

      Locating prospective interview respondents and gaining their cooperation was complicated by regulations covering issues of confidentiality, which required that the school district obtain each student's permission before releasing the student's name and address. However, the Board of Education's research office was too overworked to accept a subcontract from us to do this. As a result, we employed a guidance counselor from each school to make the initial contact and obtain permission for us to follow-up with potential respondents. This turned out to be a difficult process and by the time school ended for the academic year, only one-third of the prospective respondents had agreed. We had originally selected 483 graduates, but wound up interviewing only 110. This does not represent a low response rate (nearly everyone with whom we made contact agreed to the interview, for which they were well paid), but a low rate of success in contacting them initially. There were a number of reasons for this, the most common being that the school address for a student was wrong or because the guidance counselor was unable to make the repeated attempts that would have been necessary to contact them.

      Of the 110 graduates in this subsample, fifty-one had "won" the lottery and attended, and graduated from, their first choice career magnet. The other fifty-nine had "lost" the lottery and graduated from a comprehensive high school. All were between the ages of 19-22 years (mean age was 19.8) at the time they were interviewed. Respondents in this subsample identified themselves as follows: 46% African Americans, 3% Asian Americans, 37% Caribbean Americans, 12% Latino Americans, and 4% multiethnic. (Because there are few whites in the district schools and we thought they would be quite different from minorities in their school and post-high school experiences, we decided to select only minorities.)

The Survey

      The interview combined the qualitative richness of open-ended questions with the quantitative analysis possibilities of closed-ended questions. This format was chosen because we wanted standardized data, but discovered from our initial pilot interviews that much of the data we were seeking was too complex or threatening to gather in a closed-ended manner. The instrument was designed to capture differences in how graduates experienced their high school classes, peers, teachers, counselors, and the school as a whole; their employment history; career development; ethnic identity; and lifestyle choices such as alcohol consumption, drug use, and contraceptive precautions. Following the interview, respondents completed internal locus of control and Rosenberg's (1965) self-esteem instruments. This interview ranged from one-and-a-half to two-and-a-half hours in length. All interviews were tape-recorded for later assessment.

      Four African-American graduate students (3 female and 1 male) interviewed the respondents over a nine-month period after being trained in listening and interview techniques. Anna Allen interviewed the three Asian-American respondents and the one white respondent who fell into the sample by accident (her ethnicity had not been known when she was sampled). Most of the interviews were done by an interviewer of the same sex. Respondents were paid, including a bonus for travel time.

      All interviews were tape-recorded. In general, respondents were encouraged to talk as much as possible, and no effort was made to limit the length of these interviews. Over 95% of the questions were open-ended, and responses were recorded verbatim. Post-interview coding led to some questions having as many as 50 response categories.

      The instruments took the respondent through the years from eighth grade to the present, asking their recollection of why they chose their high school program, what their attitudes about college were at that time, what their work experience and career goals had been, what their socio-emotional strengths and problems had been, and the amount of support they received from their school staff. For the high school period, they were asked about their peer groups, the kinds of friends they had, the amount of support they got from the work that they did, and the connection of that work to school. A large number of questions dealt with the connections between school and work and the degree to which classes integrated school and work. There were also questions about community service.

      Students evaluated the amount of support they received from teachers, and evaluated themselves in terms of their greatest accomplishments and difficulties, their level of confidence, and any problems they had in school. They were asked whether they were under peer pressure to perform poorly in school, how much counseling they received from staff about college and career, and what advice they received from significant others in their family and social group about life choices. They were asked their opinion of the teachers they had and why they did or did not like them.

      They were asked why they had chosen the career that they had in mind; whether the high school gave them opportunities to have a mentor, to job-shadow, or to hear speakers about work; and generally how they felt about the amount of information that was available to help them make career decisions. They were asked to describe their present job--its strengths, weaknesses, and general characteristics; how many times they had changed jobs and why; what their future job aspirations were; and their evaluation of their work skills--their strengths and weaknesses and their performance compared to fellow workers. They were asked to describe the skills needed for their particular job, so that their responses could be related to SCANS criteria.

      Next, they were asked about their college plans in high school, and whether their parents were willing to provide financial support. In terms of college, they were asked what their grades were, the amount of homework they did, what difficulties or problems they had, and what their strengths and weaknesses as a college student were.

      In the last section, they were asked a number of personal questions about what they like to do; their sense of happiness; their sexual behavior, use of contraception, and whether they had conceived a child; their alcohol and drug use; and their history of fights, arrests, and victimization. They were also asked about their family structure, their parents, older and younger siblings, the amount of contact they presently have with their parents, the amount of stress in the family, feelings they had about the work their parents and siblings did; the family's problems, activities, and the role of punishment and violence; and the family's experience with welfare. They were asked about their marital status, their use of child care if they had children, their religious behavior and the support they received from their religious organization, and their political views and racial attitudes.

      In general, questions probed much more toward the affective side of their opinions--questions like "What do you love to do?" were asked as well as "What do you like to do?" A number of questions gave them an opportunity to talk about emotional problems and about support from various others, as well as lack of support. Given the context set up by the instrument, we are optimistic that their answers about fighting; their arrest records; their victimization; and their use of cigarettes, alcohol, and drugs are reported more accurately than in most surveys.

      There was no significant difference in the ethnic make-up of the comparison groups in the survey and life history interviews (although there are more students in the control group whose families do not use English at home). The original experimental plan was to create pairs matched on ethnicity, gender, same choice of career magnet program, eighth grade reading scores, and the junior high school they came from. This procedure provided us with too few possible pairs, so the junior high school of origin criteria was broadened to include other junior high schools in the same general neighborhood. In the final sample, lottery winners and lottery losers were similar in most important respects. Demographic analysis reveals no significant differences by sex, ethnicity, eighth grade reading scores, or parental education, but one test (seventh grade math) does favor the career magnet graduates. Measures of reported socioeconomic status (SES) show no appreciable difference between the two groups. All had entered high school in the Fall of 1988 and all had graduated by 1993. None had been in special education.

      Table 1.2 shows no significant bias between the two groups, but does show that both groups are heavily female. This is partly because our sample of good experiments included more females than males, and partly because these four career magnets emphasize careers that tend to attract female students. However, the fact that the sample is two-thirds female does not bias the comparisons of lottery winners to lottery losers since females are overrepresented in both groups.

      After interviewing them, we discovered that 11 of the 59 lottery losers had been admitted to career magnet programs within their comprehensive high schools. Table 1.2 shows a significant bias. Graduates from the comprehensive programs had lower seventh grade math standardized test scores than did the lottery winners.

Life History Interviews with 26 Graduates

      From the 110 graduates who participated in the two-hour "short" interview, we selected a subsample of matched pairs for a semistructured life history interview. We were able to successfully interview only 13 matched pairs; six others who were possible matches cited various reasons for not participating. There are no discernible differences between those not re-interviewed and those who were re-interviewed. This subsample of 26 graduates was composed of five pairs of males and eight pairs of females; four non-matching respondents were also interviewed, bringing the sample to 30.

      The life history/career development interviews were approximately four hours long and, when possible, conducted over two different days. The first two hours were devoted to a life history and the second two to tracing career development and the respondent's activity after high school. The interview guide began with general background information about the respondent and his or her family prior to entering school. It included specific questions about the family and educational experiences for each chronological year of school from kindergarten until the time of the interview. This procedure provides a complex description of how school and home interacted for each respondent to affect their career development and what meaning each graduate retrospectively attaches to these experiences. Asking each respondent specific questions about each year allowed us to look for patterns across cases.

      Each interview typically produced a transcript of slightly over one hundred pages. Transcripts were coded using an ethnographic program, "Atlas/ti," recommended to us by Matthew Miles. Judging from the number of stories of childhood trauma and descriptions of misbehavior, this interviewing procedure seemed successful in getting candid histories.

      Whenever possible the interviewers who had interviewed the respondent for the semistructured interview did the life history interview in order to take advantage of the rapport established during the first interview. Each case was coded and analyzed by the graduate student who did the interview. Cases and strategies were discussed in frequent meetings with the authors.

      Since the 30 respondents provided both the interview and the life history, it was possible to mix the quantitative analysis of the interviews with the ethnographic analysis of the life histories.

Bias

      While this is not a perfect randomized experiment, it comes much closer than traditional longitudinal studies. Students had been randomly assigned to one of two groups, either winner or loser, through the administration of a lottery. They had chosen the same programs; were in the same reading ability groups; and, in the case of the survey, shared the same talent and tenacity necessary to graduate from a high school. We think the most important attribute of this study is that it evaluates schools that are in the real world, to which students were assigned randomly. The schools are not pilots or experiments; they have been mass-produced and have passed the test of feasibility. The use of lottery admission gave us the opportunity to base our analyses on an initial random assignment of students. While we could not achieve the rigor of a laboratory experiment such as might be done in medicine, we think the experiment-based statistical analysis here, using the lottery admission to these high school programs, provides us a much stronger analysis than has been done before.

References

Bailey, T., & Merritt, D. (1997). School-to-work for the college bound (MDS-799). Berkeley: National Center for Research in Vocational Education, University of California, Berkeley.

Kemple, J. J., & Rock, J. (1996). First report on the career academies demonstration and evaluation. New York: Manpower Development Research Corporation.

Rosenberg, M. (1965). Society and the adolescent self-image.Princeton, NJ: Princeton University Press.

Stern, D., Raby, M., & Dayton, C. (1992). Career academies: Partnerships for reconstructing American high schools.San Francisco: Jossey-Bass.

CHAPTER  2
Career Magnet Graduation Rates

      Many career magnet programs have lower graduation rates than the comprehensive schools. Only 26% of the lottery winners graduated at the end of the fourth year, while 31% of the lottery losers graduated after four years. This is not because lottery winners are more likely to transfer out of the school district; 16% of lottery winners and 17% of lottery losers leave the school system. The difference is in dropouts. At the end of the third year of high school, 7% of the lottery winners had dropped out of school, compared to 6% of the lottery losers. After the fourth year of high school, 14% of the lottery winners had dropped out, and only 11% of the lottery losers had dropped out. We do not have data on the ultimate difference in dropout rates, since 44% of lottery winners and 41% of lottery losers were still in school without having earned enough credits for graduation at the end of the four years.

      Because lottery outcome is not synonymous with magnet/nonmagnet placement, the results are understated (see Appendix B). We conclude that comprehensive schools are graduating four students for every three that career magnets graduate. The career magnets' lower graduation rate and higher dropout rate are statistically significant and of considerable policy significance. Since our research reveals that the lottery winners were not academically inferior to the lottery losers, the lower graduation rate cannot be explained by a difference in academic ability.

Looking for an Answer: Why the High Dropout/Low Graduation Rates?

      There are good reasons to be surprised by this finding because there are seemingly obvious reasons why career magnets should be more successful at holding students and graduating them than comprehensive schools. Therefore, we need to look at possible reasons why the opposite could be true.

      A fundamental problem with career magnets is the conflict between offering students the best education while providing employers with qualified workers. On the one hand, students need to be educated with a curriculum that moves at their pace and at the appropriate level of difficulty; however, on the other hand, employers need workers who are sufficiently qualified to meet their needs. A student might be able to make satisfactory progress in a high school program, yet might not be able to meet the standards of the workplace where they expect to be employed after graduation. The school is forced to set higher standards to satisfy the demands of its commitment to prepare students for employment than it would if it were providing an education merely sufficient for graduation and admission to a local community college or university.

      Nowhere is the conflict more dramatic than in the area of health. Traditionally, the district's vocational high schools had prepared students for the practical nurse license. However, over time, the number of hours of practice required and the amount needed to be learned made it virtually impossible for students to pass the LPN examination by the end of high school. One solution, which is now widely used, is to replace the LPN license with the Nursing Assistant Certificate, a much more elementary certificate primarily used to qualify people to provide bedside care in nursing homes. The requirements for a Nursing Assistant Certificate can be met in high school, by on-the-job training, or by training done by the hospital workers union. Use of the Nursing Assistance Certificate is one way high schools can provide students with a "success opportunity"--a test they can pass. It qualifies students for low-level work in a hospital setting, and may be viewed as a first step in preparing a student for community college work on an LPN or RN license. Up until the last few years, hospitals had been uninterested in high schools providing nursing assistant training; instead, they wanted schools to develop rigorous academic programs for students who could become college trained RNs. At that time, there was a considerable shortage of RNs, and hospitals would have liked high schools to increase the number of students entering the college nursing pipeline. Many educators were opposed to this, however, believing that they would simply be creating another program where many students would fail.

      Dropping requirements down to the Nursing Assistant Certificate level, however, does not solve all of the problems. Programs must be sure that their students will be "safe" workers in a hospital setting. Because every career magnet that attempts to place interns or graduates is preparing students for a demanding job, the problem of sorting students never goes away.

      In areas where there are no credentials, the problem is even worse. How well-prepared must a secretary, a computer programmer, or an apprentice accountant be to not embarrass the school? There is no clear answer, but the relative payoffs to the school are clear: An annoyed employer can harm the program by not taking interns in the future or by criticizing the school among his or her colleagues, while a student who does not get an internship will not result in any loss to the school. In that situation, a rational decisionmaker will use caution and raise the bar higher to make sure that every student who is presented to the employer is a credit to the school.

      Traditionally, high schools served only the last three grades of high school, and vocational programs tended to use the first year as an "introduction to occupations" year in which students were not in a particular program. They entered a program in their junior year and received a two-year preparation. This is a common strategy still: for example, the Academies of Travel and Tourism, which operate in a number of high schools in the nation, are two-year programs and admit their students at the beginning of the junior year.

      This traditional structuring of high school was made more complicated for academic career magnets in three ways. First, moving ninth grade into high school meant creating a two-year pre-career sequence of courses before students began internships and most of their career training. The second complication arises from the fact that students are required to commit themselves to an occupation before entering ninth grade and then wait two years to test their choice with real experience. The third complication is that it often takes more than two years to complete enough high school credits to enter junior-year status. The result is that students are in a holding pattern for two or three years after having chosen a career. These complications may explain the high dropout and low graduation rates of academic career magnet programs.

What Kind of Students Are More Likely To Drop Out of Career Magnets?

      There is no easy way to answer this question. Even students in the process of dropping out are not always able to evaluate which of the various pressures was the most substantial. Our best information on this subject comes from asking, "What are the characteristics of dropouts that distinguish them from students that stay in school?" These characteristics differ slightly between career magnet programs and regular comprehensive schools, and in an unexpected way. Table 2.1 shows the eighth-grade factors that predict the graduation rate for lottery winners and lottery losers. The first two factors--eighth-grade absenteeism and middle-school grades--can be thought of as motivational. They reflect a lack of interest or a lack of motivation. The last two--their middle-school reading and math scores--measure cognitive ability.

      The hypothesis is straightforward: The comprehensive high school teaches academics and evaluates students on their academic performance; therefore, it should be difficult for a student with low academic aptitude, here measured by middle-school test scores, to graduate. In contrast, the career magnet school is preparing students for both school and work and requires multiple modes of intelligence. Students need to have good work habits, character, and commitment. The student with weaker scores but a stronger work ethic should be more likely to graduate from a career magnet than from a comprehensive high school. Table 2.1 shows exactly the opposite effect.

      The career magnet, which should be more concerned with multiple modes of intelligence, instead requires higher cognitive scores for graduation.

      This evidence shows that both poor motivation and poor scores predict failure to graduate from high school, but for the career magnet programs the effects of test scores are slightly more important. This data supports our conclusion drawn from visiting the schools that the career magnet schools are academically more demanding than the comprehensive schools. This is surprising. We would expect the dual-focused academic career magnet to utilize the students' multiple modes of intelligence, since it teaches multiple lessons; instead, it is requiring more traditionally measured cognitive ability than the comprehensive school, which is expected to evaluate students solely on academic performance.

      There is an important irony here. The present high rate of unemployment among urban youth means that the student who graduates from a comprehensive high school and does not go on to college has poor employment opportunities. The student with a high school diploma from a career magnet school may have an entry-level position available--in word processing, for example. If the career magnet bestowed a high school diploma on a student with weak cognitive ability but good work habits and word processing skills, it would help the student gain employment as an alternative to college. Yet the career magnet is setting cognitively higher standards for graduation than the comprehensive high school. Higher cognitive standards for graduation lead to a lower graduation rate for students with weak test scores.

Using the Lottery Experiments To Identify Traits of Programs with High and Low Graduation Rates

      We pursued the problem of why career magnets have lower graduation rates by looking at the graduation rates of different career magnet programs to see if programs with low graduation rates have particular characteristics. One of the few requirements placed on these programs by the board of education, and the most important for research purposes, is that these programs are required to offer half their seats to students on the basis of one of three lotteries conducted separately for students with high, average, and low reading scores. In our study, we were thereby able to create evaluations, based on the lottery as a randomized experiment and the responses to a telephone survey of school administrators, to measure outcomes of 18 programs serving students with high reading scores, 35 programs serving average students, and 39 programs serving students with low reading scores--all programs where we had data from the program administrators (some administrators refused to be interviewed). Since many programs had enough applicants at the various reading levels to permit us to create an experimental design for more than one reading group, these 92 experiments are located in only 49 programs.

      Through our analysis, we will attempt to explain the graduation rates for students of these 49 programs. For pretest data, we have seventh- and eighth-grade reading and math examination scores, grades, and attendance data for each student. Students apply to high school at the end of eighth grade, so that seventh- and eighth-grade scores predate the students' choices of high school and predate their lottery assignments.

      Our main source of independent variables is a telephone survey of program administrators in which we asked whether their particular program had various elements in its school-to-work curriculum. In this report, we will use their answers on four topics:

Analysis

      Since each reading level is a separate experiment, our analysis must be done separately for each. We focused our analysis mostly on the average group, which represents not only two-thirds of the district's students, but also the performance of typical low and moderate income minority students. We analyzed the data in two ways.

      Step A: The correlations, computed at the program level, between the extent of each career program component and the "program effect," and the performance of applicants to the program who were lottery winners compared to the performance of lottery losers among the applicants--both adjusted for seventh- and eighth-grade academic performance--were evaluated. This is the "perfect" experimental result in that it is unbiased. Being unbiased, it includes students who were randomly selected into the "experiment" and "control" groups but did not actually experience the "experiment" or "control" treatments. As already discussed, some lottery winners (29% for whom we have students outcome data) did not attend their first choice academic career magnet program, and some of the lottery losers (18.3%) received the experimental treatment because they were able to attend the academic career magnet because they were selected by the program (see Appendix B).

      Step B: Next, we computed mean test scores from the individual data file, comparing individuals who won and lost the lottery, but separating those winners who did not go to the program and also those losers who did go. These tables are no longer an unbiased experiment, but they provide a test to validate the significant program-level correlations done in Step A, and they give us our best estimate of the magnitude of the effect.

      Procedure for Step A: We compared the ability of the programs to graduate their students by computing for each program the graduation rate of the students who applied to the program and won the lottery to the graduation rates of the applicants to the program who lost the lottery. We took several steps to make the most accurate comparison between different programs. First, we compared the graduation rate for all the students who won the lottery, whether they actually entered the program or not, to the graduation rate for all the lottery losers, including even those who were picked by the program--this meant we preserved the randomness of the lottery, getting an unbiased measure of the "graduation power" of each program in comparison to the other programs the students might attend, and thus a good measure of the difference in graduation power between the different academic career magnet programs. (This technique does underestimate the differences among the programs, as shown in Appendix 2; however, it eliminates any bias caused by discarding well-qualified lottery losers getting into some of the schools and perhaps some highly qualified lottery winners choosing not to enter some schools because they were invited to attend a more prestigious school, for example.) Because the students are sorted into separate lotteries depending on test scores, we might have (if the number of applicants is large enough) two or three separate measures of the "graduation power" of each program. We found in some programs that students with high test scores who won the lottery were more likely to graduate than were high-scoring applicants who lost the lottery, but applicants with low test scores did not increase their chances of graduation by winning the lottery. For other programs, we found the opposite: The high scoring students did not increase their graduation chances by winning the lottery, while the students with low scores did.

      Second, we adjusted (using multiple regression) for any difference between lottery winners and lottery losers in seventh- and eighth-grade test scores, absenteeism, and grades. Since the lottery is random, differences should be small; however, the slight benefit of removing random error in the lottery drawing is worth computing the regression equation.

      Third, we correlated characteristics of the programs with our measures of their "graduation power," doing separate correlations for each of the three reading levels, since these are separate lotteries. Since the proper statistical significance test should use the number of programs (49) as the degrees of freedom, not the number of students (7,987), we correlated the "graduation power" of each program with the various measures of the program's management, resources, and practices. (Since these programs vary in size, we weighted the data for each program based on the number of winning and losing students using a formula suggested to Armor (1972) by Frederick Mosteller for aggregate data.) The apparent effect of each program was measured by computing a graduation rate or dropout rate, adjusted by regression for seventh- and eighth-grade standardized reading and math scores, grades, and absences. The mean for all students who applied to each program as their first choice and lost the lottery was subtracted from the mean for all first choice winners to the same programs.

Results

      At the aggregate program level, we found a negative correlation (-.565, p = .001) between graduation rates and the amount of job placement. This is a comparison of all lottery winners and lottery losers who had applied to each program. This is the proper comparison to test for significance but far too conservative to estimate the actual magnitude of the program effects. Table 2.2 shows an estimate of the effect of the higher graduate job placement rate on graduation rates of students in the average reading group based on the individual data file. In Table 2.2 and the following tables in this chapter, this is our best estimate of the effect of the extent of career placement.

      The figure in the lower right-hand corner, 43%, is the graduation rate for students who had applied to a program that did not have a strong placement component and who were not admitted by lottery. In the lower left-hand corner, 38% is the graduation rate for students who had applied to the same career magnets, won the lottery for admission, and attended the career magnet school. The difference of 5% implies that students who won the lottery were less likely than those who lost the lottery to graduate from high school within five years.

      The figure in the upper right-hand corner, 42%, is the graduation rate of students who applied to programs with a high placement rate but who lost the lottery and were not school selected. Finally, the number in the upper left-hand corner, 34%, represents the graduation rate of students who applied to these same high-placement-rate programs, won the lottery for admission, and entered the program.[1]

      The students who applied to the career magnets with high placement rates and lose the lottery have the same graduation rate as lottery losers who applied to the career magnets with low placement rates. This is a surprise. One might have expected the students who were interested in high school programs with high placement rates to be less interested in college and less interested in education generally; they would then have a higher propensity to drop out from a regular comprehensive high school. The catalogue that middle-school students use to choose programs usually does not describe post-high school employment opportunities, so it may be hard for students to choose on that basis. More likely, nearly every middle-school student plans on going to college, so whether the school offers employment after high school is irrelevant to them.

      Regardless of whether the program one applies to places more or fewer graduates, students are more likely to graduate if they do not win the lottery for admission and do not attend a career magnet program. They are also more likely to graduate if they begin high school in a comprehensive school. In addition, the fact that the difference between lottery winners and lottery losers is greatest in the first line suggests that the graduation rate from career magnet schools is especially low in those programs with a high placement rate. This pattern is confirmed by the program-level analysis: The .565 program-level correlation presented in Table 2.2 is not biased and is statistically significant.[2]

      Other findings are also consistent with Table 2.2. For example, programs that place many of their graduates in employment have a higher ninth-grade dropout rate for average reading-group students (program level r = .39, p < .02).

      Circumstantial evidence suggests possible reasons why schools that emphasize finding employment for its graduates may have fewer graduates. Table 2.3 shows that all career magnets, but especially those with high placement rates, have greater ninth-grade absenteeism. The effect of winning the lottery and attending a career magnet program with low levels of placement is to increase one's first year absences from an average of 12.6 to 14.3, a gain of 1.7 days; however, for those who use the lottery to get into programs with high placement rates, the difference is an increase from 14.4 to 17.6, a gain of 3.2 days. (To minimize error introduced by long tails in the absenteeism distribution, in this and the following tables, the means shown are, in fact, the antilog of the mean of the natural log of the number of days absent.)

      Although this table shows a large difference in days absent, the program- level correlation is not significant. We do get statistical significance when we use a related variable, however. Program heads were asked, "Some schools focus on college preparation, some focus on college preparation geared toward particular careers, and some focus on career preparation. Which one of these best describes your program: college preparatory, college preparatory with an emphasis on a career in . . ., or career preparatory?" They were then asked, "Does your program prepare students to work if they choose to upon graduation?" Finally, the program head was asked, "In addition to the high school diploma, does your program offer any special certification, license or diploma that is not offered to the rest of the school?"

      Only 10% said that they were college preparatory and did not provide students any preparation for work; another 10% said that they were purely career preparatory, not preparing students for college. Eighty percent of the programs said that they did both college and career preparation; four-fifths said they prepared students for work, and four out of seven said they offered a special certificate connected to career training. When we correlated the amount of program emphasis upon career and the number of days absent in the first year of high school, we found that those that emphasized career placement had significantly higher absenteeism (r = .33, p = .05).

      The second-year data also show high-placement programs having higher absenteeism. Table 2.4 shows a high absenteeism rate during this second year of school for students who win the lottery to attend high-placement programs, and the program-level correlation, .609, is highly significant. These results are not the result of any bias in the lottery. Lottery winners to programs emphasizing placement actually have significantly lower absenteeism in the eighth grade (r = -.382, significant at p = .03) than do lottery losers who selected the same programs, so the high absenteeism presented in Table 2.4 occurs despite having to adjust for an eighth-grade self-selection bias towards low absenteeism.

      The number of credits earned towards graduation is also quite low in both years (program-level correlation with placement is -.70 in the third year and -.58 in the fourth, both significant at p < .001). The combined effect of the high dropout rate and the inability of students to pass enough courses results in a large number of students who are unable to graduate at the end of four or five years.

      After visiting many programs, we concluded that programs emphasizing employment after high school had two problems. First, they had to set higher standards for their students, which made graduation difficult and school alienating. Secondly, they postponed career education until the third year, partly to wait for students to mature and partly to give the program time to screen out students who seemed least able to do the work for which they were to be trained.

      Why do students in career magnet programs with high placement have high absenteeism, fewer credits earned, and a lower graduation rate? We can only speculate because we have no direct data from a large enough sample of tenth-grade classrooms, but it is possible that by tenth grade, students are both bored and frustrated--bored by the relatively small amount of career content in their classes and frustrated by the high level of difficulty of those classes. Such difficult courses are likely to be used by school staff to identify those students who are most likely to be suitably qualified for internships, career-related jobs, and the advanced classes to prepare them for employment.

      It is also possible that third and fourth year students are not earning credits because the demands of career classes are too great (since the school staff is concerned that its students are well-qualified for employment) or because students are overloaded with the combination of internships, part-time work, and their schoolwork.

      Table 2.7 shows that lottery winners in programs with a strong emphasis on placing graduates are 5% more likely to transfer to another school by the end of their third year of high school than are lottery winners in programs with a low level of emphasis on job placement or no placement facilities at all. In the fourth year, the transfer rate is 5% higher (r = -.54, p = .001, table not shown).

Setting Quotas for the Program's Junior Year
      The need to select students who, in the eyes of the program staff, can meet the demands of a program's junior- and senior-year career preparation often results in dropping all but a small proportion of students from the program. One business program we visited ranked all its second-year students, whether admitted by lottery or school selection, and kept the thirty highest ranked students of the ninety who had entered the program in ninth grade. The rest became regular students of the comprehensive high school. We should not exaggerate the harm done; the students have not been relegated to a dustbin bottom track because the comprehensive high school in which that program is housed has a good reputation. We interviewed the director of another finance academy and heard an identical story; the program graduates only 34 students, the number of seats in one classroom. Other students remain in the program but do not get into internships or advanced classes. Over half of the programs we visited used some variation of this process of setting a fixed size for its junior class and then admitting enough students to make sure they had enough talented students. In the third case, the class size was limited by technological resources to only twenty. The programs all chose to admit a considerably larger number of students, guaranteeing that most students entering the program would not be allowed to finish it. The dropped students were provided the opportunity to continue in high school either by being in the program in name only and taking the same sort of courses that any other high school student would take, or by being provided an alternate set of courses in a "safety net" program. Whatever the solution, the fact remains that when admissions decisions were being made, they were made by counselors who knew what percentage of these students would finish the program to which they were being admitted. Several program heads said that there were no dropouts from their program, but they were most likely referring to the third and fourth years, when their student body had already been drastically winnowed down.

      One program claimed to allocate the same number of seats in each grade, ninth through twelfth. Asked what their dropout rate was, the program head announced that it was zero; it had to be or else the upper classes would be underenrolled. Clearly there must be some exaggeration here, but they were under self-imposed pressure to hold on to as many students as possible from ninth through twelfth grades.

      In our interviews (Flaxman, Guerrero, & Gretchen, 1997; Heebner, Crain, Kiefer, & Si, 1992), we met a number of students who came to realize that they had no interest in a particular career once they had learned a little more about it: "I thought I would like accounting, but it turned out just to be all math. I wound up hating it." Other students talked about planning a different career while they were in high school studying for a career in which they had already lost interest. It seemed that many of these students were able to learn a lot about careers and themselves in the process of making a change. We often concluded that being trained for what turns out to be the wrong career ended up being a better educational experience than they would have had in a comprehensive high school. This does not mean, however, that the students themselves understand this as they are going through the turmoil of adolescent career-identity-formation at the same time.

      Moreover, it may take students more than two years to reach junior-year status, when internships and the academic career focus begins. As a result, and this is probably true of other urban districts, reliable graduation statistics require waiting until students have had the opportunity to be in high school seven years. The number graduating "on time" at the end of four years is small--only about 40% graduate that quickly. This mirrors the pattern in college, where graduation rates can be estimated only after students have had time to drop out and return, to carry reduced schedules because of working, or to take the high number of courses required for graduation in a particular major. Many college programs frankly admit that it is not possible to complete all the requirements in eight semesters. This same pattern is appearing in high school as academic standards are being continuously raised by state legislatures and through pressure from the federal government. Raising the performance bar by requiring students to pass multiple tests for graduation and (in the programs we studied) adding more coursework and internships slow down graduation. Students are not rushing to graduate so that they can take high paying jobs at the age of 18--there are very few such jobs--but there are more part-time jobs for high school students, which also tends to delay graduation. Furthermore, the increased pressure on all students to graduate from high school means that students who fail courses are under great pressure to remain in school and retake classes, which can also lead to added years of high school.

      One factor that may increase dropouts in career magnet programs is the stretching of the number of years between entering high school and achieving junior-year status, when internships and career courses become available. The student who cannot accumulate enough credits or pass enough examinations to move quickly to junior-year status is at high risk of dropping out, and this could be a major explanation for the high absenteeism of students in the first two years of the career magnet schools.

      Most educators, including program administrators, showed a genuine commitment to all of their students, and this came through in our interviews. They were proud of their low dropout rate, although they often calculated the rate by looking only at seniors, rather than the entire period of time students spent in their program. Those staff and administrators who ran a program with a low four-year dropout rate were proud of their accomplishment. On the other hand, the program administrator of the finance program discussed above justified his willingness to discard two-thirds of the students admitted to the program at the end of their second year by arguing that they probably were not very interested in finance but used it as a device to get into a better comprehensive high school than the one in their own neighborhood. This comment, which is part truth and part rationalization, was heard often enough to make it clear that program administrators knew that dropping students is a moral act. Others talked about taking risks such as selecting a student for an internship despite his or her sometimes poor performance in the past. We watched teachers select the weakest student in a group to be its leader in an effort to bolster the student's self-esteem.

      While some programs take pride in their demanding standards, others take an equal amount of pride in holding students in school who might otherwise have dropped out. The director of a veterinary science program said that providing students with an opportunity to take care of animals gave them a strong incentive to stay in school; although sending a small number of students to high-quality colleges was important to the staff, it was clear that helping students graduate was also a clearly defined part of the program's mission.

      The evidence of trying to keep students and the discomfort in talking about dropouts is important because it shows that policymakers have a large reservoir of commitment to draw upon, if they can only devise strategies that will reduce the cost of keeping the weak students. All program administrators would like to find a way to place their most gifted students in industry or a good college and still encourage their weaker students to remain in school.

      Employers are being rational when they demand that schools send them only their best students. Students with stronger academic skills generally make better employees. Since career magnet programs must evaluate students in order to decide which ones qualify for internships, it seems inevitable that the academically stronger students will have the advantage, and the academically weaker students will be passed over. While this does not mean that academically weaker students must drop out of career magnet programs, it does mean that a special effort has to be made to keep them in the school. Preference for high-achieving students may be inevitable, but a low graduation rate for low-achieving students is not.

      How do students who do not make the grade finish their schooling? We were surprised at the complexity of the answer to this question. In some programs lodged in comprehensive schools, program administrators stated that they were not permitted to expel a student from school simply because they have not been selected to stay in the program. Even though the student had come from a distant neighborhood, he or she would remain in the school rather than return to the neighborhood school; however, other program administrators in the same situation said they were required to return them to their neighborhood school.

      Certainly, forcing students who had not been selected for career training after their second year in high school to return to their neighborhood schools would be expected to increase the dropout rate. Students who change schools are more likely to drop out, and this is as true for career magnets as for students in other schools in the district. In situations like the finance academy discussed above, where two-thirds of the students do not remain in the actual program after their sophomore year, those who remain in the same building will have friends in the classes they take after leaving the program--either comprehensive students they knew from previous classes or extracurricular activities, or other dropouts from the program. Thus, their chances of staying in school are higher than if they were to change schools.

      Buildings made up entirely of career magnet programs have a different problem because they have no place to send students who have been dropped from their program. Some of these programs do not drop weak students; instead, they modify the program to accommodate the students. For example, one business program retains its weak students but assigns them to in-house internships, where they work as clerical staff within the school. Obviously, this kind of internship has less of an emotional lift for students, but it avoids dropping them from the program.

      Other programs successfully use technology to keep students in school. A good example of the latter is an accounting program in a career magnet school that has high standards and expects nearly all of its students to go on to college. Students who have difficulty with accounting are "pulled out" into a remedial accounting class in which computers are used to help students understand the abstraction behind more difficult mathematical formulae. According to the program administrator, this enables students to catch up and join the rest of the class by the following year.

      As students reach these frustrating last years of high school, trying hard to accumulate enough credits for graduation or to pass mandatory examinations, their career counselor may be giving them useful advice and moral support, perhaps urging them to stay in school by pointing out that there are careers in this field that do not require high-level academic skills. Table 2.9 shows the lower dropout rate of students who won the lottery to enter schools with more career counseling compared to those who won the lottery to enter a program that had less.

Adding an Academic Career Focus in the Early Grades
      Some career magnets have tried to maintain student interest during the lower years of high school by creating introductory courses in keyboarding and computer work. Others have devised ingenious ways to incorporate the academic career focus into elementary courses (such as teaching aerodynamics with a strenuous paper-airplane competition). Others introduce career-related material into some of their academic ninth- and tenth-grade courses. Some use a variety of guest lectures and visits to work sites as a way of introducing students to their future careers. These techniques seem promising. We have no hard evidence that they succeed, but they merit a more thorough evaluation than we were able to do.

Career Counseling
      Although our only positive findings that career counseling is valuable was for students reading below grade level in the seventh grade, we believe career counselors are also valuable for the average student as well. If students feel from their first class in a field that that career is not what they thought, a career counselor can reassure them that their original interests in their field were not based on misconceptions. If they are frustrated by not having enough hands-on career work, a counselor may be supportive and encourage them to do career-related work after school. If students really are locked into a career they are uncomfortable with, a career counselor can encourage them to look at a particular career as a short-term plan, a way of supporting themselves while they go to college. Since most high school students hope to go to college, this is a strategy with wide utility; a number of students we interviewed talked about using their present career training as a way of paying their way through school, or as a back-up career if their long-term goals do not work out. Some of the teachers and counselors in career magnets encourage this sort of thinking.

Shifting Programs
      Sometimes a school can simply move the student into a program with a different focus. Students who looked forward to a health career but are missing school because they cannot stand the sight of blood might, for example, do well in a program that trains them to work in a medical office.

Incentives: What Governments and Markets Can Do
      Students of government and educational policy usually complain about the overregulation of schools, but in the case of career magnets, the role of government has often been helpful, and there are certainly examples of how schools and programs, when left alone, fail to serve all their students well. In many cases, government action may be beneficial in an unexpected way. In the past two decades, critics of government have argued that the market is a better source of incentives because clients can "vote" for programs that seem most helpful to them. In this section, we mention a few examples of governmental and market incentives that have affected, or could affect, career magnet programs.

      One of the factors intended to work to the advantage of low-achieving students is the Perkins Act's requirement that the schools receiving vocational education money educate their students in "all aspects of the industry." The phrase, inserted in the bill as a result of the lobbying efforts of the Center for Law and Education, was intended to make sure that vocational students were prepared not only for the bottom rung of a particular industry but were also taught about the technical and managerial side of the work as well.

      While every bricklayer cannot become a private masonry contractor, the school should at least not stand in the way of their students' effort to move up the career ladder of an industry. Perkins has had no effect on career magnets because these programs receive no vocational education funds; if they did, each career magnet would feel pressed to make sure that it had programs covering a wide range of activities within its chosen industry.

      In the absence of federal pressure, student applicants have exerted market pressure. Many choose career programs with a precollegiate flavor. In response, some program administrators and staff have developed programs or sets of programs that hold out the promise of college to all while at the same time making sure that they have courses appropriate to a wide range of student abilities. The easiest way of doing this is to teach students about a range of careers within a single broad industry. Thus, for these programs, focusing on all aspects of an industry can be used as a recruiting tool, offering preparation for high-quality post-college employment. Such a broad academic career focus can also benefit students who are not skilled enough to obtain a college degree or a high-quality internship. If schools provide information and internships in some of the intermediate and lower-level positions in their field, students will be able to begin at the bottom even if they cannot jump-start their careers by obtaining a college degree. This also means that programs that train students in all phases of an industry will necessarily have a "safety net" career line of training that could keep potential dropouts in school.

      A stress on all phases of an industry also encourages students to think about related fields that require different skills. Industries can accommodate a wide range of student interests. For example, even a highly technical field such as engineering has room for people with strong interpersonal skills and less interest in analytic work.

Incentives: School Size
      When some entire high schools became career magnet buildings, the schools became large enough to accommodate a group of related programs, and some school planning committees took advantage of the opportunity to teach all phases of an industry. The schools that did this seem to be the most successful. The one total career magnet school that is an exception, creating four unrelated programs within the same school building, has had more difficulty holding down its dropout rate. In another example, local and state governments required programs to take a quota of students with low reading test scores; this created an incentive for some programs to develop anti-dropout strategies, including creating safety net alternatives.

      When the city Board of Education agreed to allow comprehensive schools to create career magnet programs, the incentive was to keep the programs small, since schools usually have little empty space in which to work. As a result, comprehensive schools were unlikely to develop dual-focus or safety net programs.

Incentives: Targeted Budgets
      School district administrations (and, more commonly, the state or federal government) create incentives by providing separate streams of funding dedicated to specific tasks or specific groups of students. For example, a large number of high schools provide a special education version, or non-English language version, of the career magnet program created for regular students. We also noted that the computer-assisted accounting program grew out of a special district fund for reducing dropouts.

Conclusions

      Our most important conclusion is that these career magnet high schools have a lower graduation rate than do comprehensive high schools. We have uncovered three explanations for the lower graduation rate in the career magnet schools:

      We have identified three strategies that seem to reduce the dropout rate: First, and most important, is the creation of a safety net that can catch those students who are dropped from a program. Second, the dropout rate seems to be lower when students are involved in individual and group projects. Third, dropout rates seem to be lower in schools that devote more resources to career counseling.

      While all three efforts are important, the creation of the safety net is most important because of the large number of students who are dropped from programs--in fact, if not in name--and have no other program to enter.

The Lottery and Accountability

      The area's public high schools are loosely coupled to higher levels of government. This research project has found that there are important roles for the government to play. First, gathering statistics is important. The fact that there are good statistics on the number of students passing required tests, and poor statistics on graduation rates for each program is an important reason why a great deal of attention is paid to the first and a lot less to the second. In the absence of good data, each program administrator we interviewed assumed that he had a relatively high graduation rate compared to other schools. Like Lake Woebegone children, every program administrator thought his or her program was above average.

      The lottery admission strategy provides an excellent opportunity for administrative oversight of the program. It is rare in education to have a clear "bottom line" accounting system to determine the success of a particular school. The lottery provides precisely that opportunity. The fact that academic career magnet applicants who win the lottery are more likely to succeed in college (at least, this is true for the four career magnet schools we studied in detail) indicates that these programs are successful. The lower graduation rate of the students who win the lottery as opposed to those who applied to the same programs and lost the lottery presents an opportunity for higher-level administrators to ask for action. As with most other school districts in this country that serve low-income minority areas, the district's schools are often indicted for the poor test scores of their students while, in fact, they may be doing an excellent job of "adding value" to the performance of their students. Conversely, schools that serve a middle class nonminority population may brag about the high rate of graduation and college placement among their students while actually providing these students with little more than they brought with them from home.

      There are other areas where higher-level administrators might play a role. For example, this research project has found that some schools succeeded because they had linked multiple programs. A school board planning to stimulate the growth of career magnets could provide technical assistance on the value of this approach. The central office can also be helpful by developing a student record system that could be used to evaluate the effectiveness of schools-within-a-school by recording the name of both the school and the program in which a student is enrolled. This would allow the administration to identify successful individual programs within an otherwise poorly performing school, or vice-versa.

Incentives and Independence: Career Magnets as Charters

      Some of the strength of career magnet programs derives from the fact that they are much like charter schools. Program directors have the opportunity to fulfill their own vision of what a high school experience should be like. They do not have as much control as they need or would like, especially in recruiting staff, but they have control of their curriculum in important ways. They have "ownership" in a way that other educators--even high school principals--do not. As a group, program heads are ambitious, know they are competing for students, and are committed to creating exciting educational environments.

      But it is apparent from the schools' experiences here that the free-market model has its downside. Programs do compete for students; they "advertise" in the school districts high school directory. But they compete for quality, not quantity. Programs want large numbers of applicants so they will have a large pool to pick from when they select half of their entrants. Similarly, they also want a large entering class of ninth graders so that they will have a large pool from which to select its small junior class. Program heads do not attempt to "maximize their profits" by increasing enrollments. Instead, they maximize their satisfaction if they can feel they have "made a difference" with a group of students, preferably talented, whom they know personally. Because there is little incentive for program heads to increase the number of students who complete their program, the system seems to expand mostly by new programs being created. There are over 130 career magnet programs and several hundred other types of programs. The impersonality of the traditional high school is surprising; we interviewed teachers who were nominated by students as their "most influential adult" in school, only to find that the teacher was unable to recall the student at all. The same is true in the larger career magnet programs, which may have several hundred students. All this suggests that the smaller career magnet programs should be more successful because the program head will have more personal contact with all the students, increasing both the students' sense of being supported and the program head's sense of satisfaction. In Chapter 4, Sullivan and Little find evidence supporting this hypothesis.

The Seven-Year Curriculum

      The large number of older students in high school presents some important challenges to the school. Given the high unemployment rate among disadvantaged youth and the rigorous academic standards applied in the these high schools, it is only logical that more and more students will be using a fifth, sixth, or seventh year to complete their high school requirements. It is not clear to us whether this situation has been recognized as either an opportunity or a problem. In our discussions with school administrators, the role of the high school as a seven-year school has not come up. The issue should be put on the public agenda of secondary education.

References

Armor, D. J. (1972). School and family effects on black and white achievement: A reexamination of the USOE data. In F. Mosteller & D. P. Moynihan (Eds.),On equality of educational opportunity (pp. 168-229). New York: Random House.

Flaxman, E., Guerrero, A., & Gretchen, D. (1997). Career development effects of career magnets versus comprehensive schools(MDS-803). Berkeley: National Center for Research in Vocational Education, University of California, Berkeley.

Heebner, A., Crain, R. L., Kiefer, D. R., & Si, Y-P. (1992). Career magnets: Interviews with students and staff (MDS-386). Berkeley: National Center for Research in Vocational Education, University of California, Berkeley.

CHAPTER  3
The Academic Effects of Career Magnets

      Most American high schools have long been structured on the premise that academics and vocational education should be kept separate. That model is rapidly breaking down. With skilled blue-collar jobs moving overseas and the service sector growing, working class jobs require higher levels of literacy. A new movement is endorsing that--even for the college-bound--the right approach is "contextualized learning"--that is, the integration of career and academic preparation (Berryman & Bailey, 1992; Resnick, 1987). Others argue that the high school curriculum has no time in its college preparatory track for non-academics--that college-bound students are losing the achievement race with Europe and Asia and that even students not destined for college need higher levels of literacy and numeracy in order to work in the service sector. The issue is time--the school day is a pie of a given size, and reallocating a larger portion to careers cuts into academics. These critics have a point: The great improvement in cognitive performance of the American population over the last century has been largely the result of increased school attendance and, hence, more years of "time on task."

      Proponents of school-to-work programs in high school argue that there are "slack resources" in school because adolescents are not motivated by traditional academic programs and do not spend much of their time learning; anything that shows students a connection between academic learning and their future will increase their interest in school and take time away from television and sleeping in class. It seems unlikely that this debate has any simple answer. It is more likely that there are some ways in which a school-to-work program encourages higher academic performance, and others in which it is harmful.

      We have been provided a unique opportunity to measure the impact of academic career integration on academics. We have test score data along with descriptions of the programs that students applied to for almost eight thousand students who were randomly divided into career magnet and regular comprehensive high school programs. This is the largest test ever done of an educational program using random assignment.

Methods and Data

      Administrators were asked to base their responses only on the students in their academic career magnet program, excluding students in other academic career magnet programs in the same building or in the school's regular comprehensive program, and students in any ESL/bilingual or special education programs. Of the 59 programs for which we had acceptable numbers for analyzing the experiments, only 49 were surveyed. One program had closed, and nine program heads refused to be interviewed.

      One goal of the survey was to determine how successful academic career magnets have been in integrating academic and vocational education. We concluded that programs varied greatly not only in the particular type of academic career focus chosen but also in the extent to which they had implemented it.

      The minimum requirement for graduation is passing regular reading and math examinations, or one or more advanced versions of those examinations. Our data file contains these scores and also SAT Verbal and Math examination results. Students, either at their own discretion or that of their counselor, may take these examinations in any of the years covered by our Student Records File. As a result, the number of students in any one of the three reading levels taking a particular examination in a specific year may be small, even though we have data on 7,987 students who applied for admission to 49 programs in the Fall of 1988.

      Step A: The first is a zero-order Pearson correlation, computed across the 49 programs between the extent of a career program component and the "program effect," and the performance of applicants to the program who were lottery winners compared to the performance of lottery losers among the applicants, both adjusted for seventh- and eighth-grade academic performance. This is the "perfect" experimental result in that it is unbiased. Being unbiased, it includes students who were randomly selected into the "experiment" and "control" groups but did not actually experience the "experiment" or "control" treatments. As already discussed, some lottery winners (29% for whom we have student outcome data) did not attend their first choice academic career magnet program, and some of the lottery losers (18.3%) received the experimental treatment because they went to the academic career magnet. This raises the possibility, admittedly slim, that a significant correlation is the result of a difference between the so-called "experimental" and "control" subjects who, in fact, were not actually in the treatment and control groups (i.e., we might have students who won the lottery and went to a highly selective school or to their comprehensive school and who strongly outperformed students who lost the lottery but actually went to the academic career magnet program!). Even so, a significant correlation between a program having a particular characteristic and the program having a high or low examination score is the strongest possible evidence that the relationship is present in these programs.

      Step B: To estimate the actual magnitude of the effect (and as a check for the possibility of the assignment errors discussed above creating a false positive finding), we computed mean test scores from the individual data file, comparing the individuals who won and lost the lottery, but separating those winners who did not go to the program and also those losers who did go. These tables are no longer an unbiased experiment, but they provide a measure of the size