To Strengthen Teaching: An Evaluation of Teachers Institute Experiences

CONTENTS OF PUBLICATION

  1. Executive Summary
  2. To Strengthen Teaching: An Evaluation of Teachers Institute Experiences1
  3. Notes
  4. Appendices:

Tools for this Publication:

Notes

  1. This report was prepared for the Yale-New Haven Teachers Institute by Rogers M. Smith, Christopher H. Browne Distinguished Professor of Political Science at the University of Pennsylvania, with the assistance of Dr. Ellen Kisker, Twin Peaks Partners LLC; Jing Zhang, Emily Vince, Cris Malino, Jaime Waters, and You-Liang Deng of the Yale-New Haven Teachers Institute; Catherine Carter of the Teachers Institute of Philadelphia; Christopher P. Dean and David Seidenfeld of the University of Pennsylvania Graduate School of Education; Carla Medalia and Allison Sullivan of the Penn Sociology Department; and Timothy Weaver of the Penn Political Science Department. Special thanks go to Dr. Catherine McCaslin, Director of Research, Assessment, and Student Information of the New Haven Public Schools, and her staff for generous, skilled, and invaluable assistance.
  2. Few dismiss the older view, elaborated in the works of James S. Coleman and Christopher Jencks, that socioeconomic backgrounds strongly affect student achievement (Coleman, J. S. et al. Equality of Educational Opportunity. Washington, D.C.: Government Printing Office [1966]; Jencks, C. et al. Inequality: A Reassessment of the Effects of Family and Schooling in America. New York: Harper & Row [1972]). But evidence suggests quality teachers can succeed with students of all backgrounds. The most widely cited study on the importance of teacher quality is Sanders, W. L. and Rivers, J. C., "Cumulative and Residual Effects of Teachers on Future Student Academic Achievement," University of Tennessee Value-Added Research and Assessment Center, Knoxville, Tennessee, 1996; see also Rivers, J.C. and Sanders, W.L., "Teacher Quality and Equity in Educational Opportunity: Findings and Policy Implications," in L.T. Izumi and W.M. Evers, eds., Teacher Quality. Stanford, CA: Hoover Institution Press Publication #505 (2002). Similar conclusions are reached in H. Wenglinsky, "How Schools Matter: The Link Between Teacher Classroom Practices and Student Academic Performance," Educational Policy Analysis Archives 10, no. 12 (2002); Hanushek, E. A. and Rivkin, S. G., "How to Improve the Supply of High Quality Teachers," in D. Ravitch, ed., Brookings Papers on Education, Washington, D.C.: Brookings Institution Press (2004); Bill and Melinda Gates Foundation Staff, "High-Performing School Districts: Challenge, Support, Alignment, and Choice," Gates Foundation, June 2005, 17, at http://www.heartland.org/pdf/18328.pdf, accessed June 23, 2008; Kane, T. J., Rockoff, J.E., and Staiger, D.O., "What Does Certification Teach Us About Teacher Effectiveness? Evidence from New York City," March 2006, 6, at http://gseweb.harvard.edu/news/features/ kane/nycfellowsmarch2006.pdf, accessed December 29, 2008.
  3. Desimone, L.M., Porter, A.C., Garet, M.S., Yoon, K.S., and Birman, B. F., "Effects of Professional Development on Teachers' Instruction: Results from a Three-year Longitudinal Study," Educational Evaluation and Policy Analysis 24: 82 (2002).
  4. The value of content-area knowledge is stressed in e.g. Monk, D. H., "Subject Matter Preparation of Secondary Mathematics and Science Teachers and Student Achievement," Economics of Education Review 13: 125-145 (1994). Researchers who stress the importance of pedagogy acknowledge the comparable importance of content knowledge. See Darling-Hammond, L. and Youngs, P., "Defining 'Highly Qualified Teachers': What Does 'Scientifically-Based Research' Actually Tell Us?" Educational Research 31: 18-19 (2002). See also Frome, P., Lasater, B., and Cooney, S., "Well-qualified Teachers and High Quality Teaching: Are They the Same?" Atlanta, GA: Southern Regional Education Board, April 2005, 2; Odden, A. and Kelly, J.A., "Strategic Management of Human Capital in Public Education," Madison, WI: Consortium for Policy Research in Education, Wisconsin Center for Educational Research, June 2008, 11, 13.
  5. The impact on student achievement of teachers with better basic skills, and the impact of teachers' expectations, is documented in Ferguson, R.F., "Teachers' Perceptions and Expectations and the Black-White Test Score Gap" and “Can Schools Narrow the Black-White Test-Score Gap?" in C. Jencks and M. Phillips, eds., The Black-White Test Score Gap. Washington, D.C.: Brookings Institution Press (1998), 273-374; see also Wayne, A. J. and Youngs, P., "Teacher Characteristics and Student Achievement Gains: A Review," Review of Educational Research 73: 89-122 (2003).
  6. Ferguson, "Teachers' Perceptions and Expectations and the Black-White Test Score Gap," 273-317; Ferguson, R. F. and Ladd, H.F., "How and Why Money Matters: An Analysis of Alabama Schools," in H. F. Ladd, ed., Holding Schools Accountable: Performance Based Reform in Education, Washington, D.C.: Brookings Institution Press (1996); Kannapel, P. J. and Clements, S. K. with Taylor, D. and Hibpshman, T., "Inside the Black Box of High-Performing High-Poverty Schools," Lexington, KY: Prichard Committee for Academic Excellence, February 2005, 3, 15; Frome et al., "Well-qualified Teachers and High Quality Teaching: Are They the Same?" 1, 4-8.
  7. Kannapel et al., "Inside the Black Box of Higher-Performing High-Poverty Schools," 19; Raffini, J. P., Winners Without Losers: Structures and Strategies for Increasing Student Motivation to Learn. Boston: Allyn and Bacon (1993), esp. 245-247; Bill and Melinda Gates Foundation Staff, "Challenges to Teaching Capacity in New and Redesigned Schools," Gates Foundation, 2006, 4.
  8. Raffini, Winners Without Losers, esp. 245-247. Stipek, D. J., Motivation to Learn: Integrating Theory and Practice, 4th ed., Boston: Allyn and Bacon (2002); Frome et al., "Well-qualified Teachers and High Quality Teaching: Are They the Same?" 5-8.
  9. Goe, L. and Stickler, L. M., "Teacher Quality and Student Achievement: Making the Most of Recent Research," Washington, D.C.: National Comprehensive Center for Teacher Quality, March 2008; Kane, et al, "What Does Certification Teach Us About Teacher Effectiveness? Evidence from New York City," 22-24, 26.
  10. Arzi, H. J. and White, Richard T., "Change in Teachers' Knowledge of Subject Matter: A 17-Year Longitudinal Study," Science Education 92: 246 (2008).
  11. Desimone et al., "Effects of Professional Development on Teachers' Instruction: Results from a Three-year Longitudinal Study," 105; Shkolnik, J., Song, M., Mitchell, K., Uekawa, K., Knudson, J., and Murphy, R., "Changes in Rigor, Relevance, and Student Learning in Redesigned High Schools: An Evaluation for the Bill & Melinda Gates Foundation," Washington D.C.: American Institutes for Research, April 2007, 25.
  12. Desimone et al., "Effects of Professional Development on Teachers' Instruction: Results from a Three-year Longitudinal Study," 83; Corcoran, T. B. "Helping Teachers Teach Well: Transforming Professional Development," CPRE Policy Briefs, RB-16-June-1995. Philadelphia: Consortium for Policy Research in Education (1995). As one Houston teacher put it, "My experience with most professional development is that I dread it. They put me to sleep — they're mind numbing." Lorence, J. and Kotarba, J., The Houston Teachers Institute: Goals Accomplishments, 1999-2002. Houston: Houston Teachers Institute, 2003, 75. See also Odden and Kelly, "Strategic Management of Human Capital in Public Education," 5, 20.
  13. Desimone et al.,"Effects of Professional Development on Teachers' Instruction: Results from a Three-year Longitudinal Study," 82, 102; Yoon, K.S., Duncan, T., Lee, S. W-Y., Scarloss, B., and Shapley, K. L., "Reviewing the Evidence on How Teacher Professional Development Affects Student Achievement," Issues and Answers Report, REL 2007-No. 033, Washington, D.C.: U. S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest, October 2007, 5; Blank, R. K., Alas, N., Smith, C., "Does Teacher Professional Development Have Effects on Teaching and Learning? Analysis of Evaluation Findings from Programs for Mathematics and Science Teachers in 14 States," Washington, D.C.: Council of Chief State School Officers, February 2008, 1, 3, 15-19, 26; Goe and Stickler, "Teacher Quality and Student Achievement: Making the Most of Recent Research," 6.
  14. Desimone et al.,"Effects of Professional Development on Teachers' Instruction: Results from a Three-year Longitudinal Study," 82, 86, 102; Blank, et al., "Does Teacher Professional Development Have Effects on Teaching and Learning?" 3, 5; Kannapel et al., "Inside the Black Box of High-Performing High-Poverty Schools," 3, 13; Gates Foundation Staff, "High-Performing School Districts: Challenge, Support, Alignment, and Choice," 17.
  15. Desimone et al.,"Effects of Professional Development on Teachers' Instruction: Results from a Three-year Longitudinal Study," 82, 85, 102; Blank, et al., "Does Teacher Professional Development Have Effects on Teaching and Learning?" 3, 5; Gates Foundation Staff, "High-Performing School Districts: Challenge, Support, Alignment, and Choice," 17-18.
  16. Desimone et al.,"Effects of Professional Development on Teachers' Instruction: Results from a Three-year Longitudinal Study," 102, express surprise at not finding support for the importance of duration. Most scholars do, including Blank et al., "Does Teacher Professional Development Have Effects on Teaching and Learning?" 3, 26; Gates Foundation Staff, "High-Performing School Districts: Challenge, Support, Alignment, and Choice," 18; Kannapel et al., "Inside the Black Box of High-Performing High-Poverty Schools," 13; Yoon et al., "Reviewing the Evidence on How Teacher Professional Development Affects Student Achievement," 5; Goe and Stickler, "Teacher Quality and Student Achievement: Making the Most of Recent Research," 6.
  17. Desimone et al.,"Effects of Professional Development on Teachers' Instruction: Results from a Three-year Longitudinal Study," 82, 86,102; Blank et al., "Does Teacher Professional Development Have Effects on Teaching and Learning?" 1, 5, 27; Gates Foundation Staff, "High-Performing School Districts: Challenge, Support, Alignment, and Choice," 18; Odden and Kelly, "Strategic Management of Human Capital in Public Education," 26.
  18. Gates Foundation Staff, "High-Performing School Districts: Challenge, Support, Alignment, and Choice," 10, 13, 18; 20; Goe and Stickler, "Teacher Quality and Student Achievement: Making the Most of Recent Research," 6.
  19. Blank et al., "Does Teacher Professional Development Have Effects on Teaching and Learning?" 1, 5, 26-27, Gates Foundation Staff, "High-Performing School Districts: Challenge, Support, Alignment, and Choice," 10; Kannapel et al., "Inside the Black Box of High-Performing High-Poverty Schools," 13; Yoon et al., "Reviewing the Evidence on How Teacher Professional Development Affects Student Achievement."
  20. Yoon et al., "Reviewing the Evidence on How Teacher Professional Development Affects Student Achievement."
  21. Desimone et al.,"Effects of Professional Development on Teachers' Instruction: Results from a Three-year Longitudinal Study," 105.
  22. Gates Foundation Staff, "High-Performing School Districts: Challenge, Support, Alignment, and Choice," 18.
  23. Pittsburgh public schools have experienced severe and continuing declines in student enrollments in the decade since the Pittsburgh Teachers Institute was established.
  24. Suggesting the value of collective participation, one Philadelphia teacher stated that initially she was the only teacher from her school enrolled in the program and that "I'm so happy now there are four. It makes the level of conversation in your building rise." Popp, T., "Teaching Those Who Teach," The Pennsylvania Gazette 106: 21 (2008).
  25. By 2008 the total number of New Haven teachers who had been Fellows was 657.
  26. To use a political science analogy, it would be similar to trying to evaluate the comparative success of "winner take all" and "proportional representation" presidential primary systems by randomly assigning winning candidates to the two different parties. Little could be learned about the topic in question.
  27. Yoon et al., "Reviewing the Evidence on How Teacher Professional Development Affects Student Achievement," 5.
  28. "Principal Recognition Accorded to the Yale-New Haven Teachers Institute," New Haven: Yale-New Haven Teachers Institute, at http://teachersinstitute.yale.edu/brochures/A9.html.
  29. "A Progress Report on Surveys Administered to New Haven Teachers, 1982-1990," New Haven: Yale-New Haven Teachers Institute, 1991, at http://teachers.yale.edu/publications/index.php?page=ReportsEvaluations.
  30. The study can be found at http://teachers.yale.edu/publications.
  31. Haslam, B. M., Rouk, U., with Laguarda, K. G., "Professional Development with the Yale- New Haven Teachers Institute Model: Impact, Lessons, and Future Prospects," Policy Studies Associates, 2006, 45-46, 52, at http://www.policystudies.com/studies/professional/ynhti.html, accessed June 23, 2008.
  32. All percentages reported here are percentages of the Fellows who actually responded to particular questions. This section reports data from the Teacher Surveys on Seminar Experiences conducted in Houston, New Haven, and Pittsburgh from 2003-2008 and Philadelphia in 2006-2008. Because the overall completion rate for these surveys was over 98%, percentages of all Fellows surveyed, whether or not they responded to a particular question, are only slightly smaller. Chart figures are rounded to whole numbers; graph bars are accurate to one decimal place. All the survey data reported here can be obtained through a link at the Yale National Initiative Web site, http://teachers.yale.edu.
  33. Because the numbers of teachers in particular categories were sometimes low, and because some teachers were Fellows multiple times, standard tests for statistical significance are not appropriate. In any case, the variations were not large, generally less than 10 percentage points.
  34. In the National Demonstration Project, more experienced teachers, presumably more familiar with alternative forms of professional development, rated the overall program especially highly. Teachers with Master's as well as Bachelor's degrees were more likely than those only with Bachelor's degrees to say the seminars raised their expectations of their students, especially in the humanities. Though teachers in the physical sciences rated their seminar experiences highly, their ratings were slightly less positive than those provided by teachers in the humanities, social sciences, and other areas. For these and other variations found in the evaluation of the National Demonstration Project, see Huang C., "Conclusion of Further Analysis," research memo available at http://teachers.yale.edu.
  35. See especially Bransford, J. D., Brown, A. L., and Cocking, R.R., eds., How People Learn: Brain, Mind, Experience, and School. Washington, D.C.: National Research Council; executive summary available at http://www.nap.edu/html/howpeoplel/es.html; and Cotton, K., "Teaching Thinking Skills," NW Regional Educational Laboratory, available at http://www.nwrel.org/scpd/sirs/6/cu11.html.
  36. Overall statistics are drawn from the National Demonstration Project Survey of Fellows' Unit Use, Table 111. Data on relative ratings by teachers in different subject areas are drawn from Tables 311 and 321. All tables available at the Yale National Initiative Web site, http://teachers.yale.edu.
  37. A complete set of these comments is available at http://teachers.yale.edu. One teacher commented that whereas the previous year, only 4 out of 20 regularly completed all reading and writing assignments, with the Institute unit those numbers were reversed. Another reported that students showed detailed knowledge of a unit involving films a year and half after it was taught. An elementary teacher said that returning fourth graders immediately asked, "Are we going to do an Institute unit?"
  38. These team efforts sometimes involved classes, sometimes projects for student groups. One teacher noted that "In my school, we had projects generated by the seminars of the Institute involving several teachers not all of them Fellows. Science is particularly prone for projects with broad participation of students and staff." Another reported that "we have formed a number of teams, often involving teachers who never were Fellows. These teams have allowed Institute material to benefit many, many children other than those in classes taught by Institute teachers." A third stated, "My school has become a strong YNHTI center school with a Fellow involvement of at least 40% of the faculty. Not only do we write curricula but we developed a summer school program that utilized written units by other Fellows."
  39. Referring to Connecticut state curriculum and standardized tests, various teachers stated that although their units are not designed to improve standardized test scores, the units did "incorporate CAPT activities"; satisfied "the CMT objectives and curriculum requirements"; became "a feature in the school's Science and English curriculum" for years; added "depth" to topics that were "already part of the district's curricula"; and integrated easily into "the math program that is used at my school" and the "mandated 'literacy block' time."
  40. Drawn from Brown, J., "Memo on Retention Comments in Fellows' Questionnaires, 2000-2005," 2008, available at the Yale National Initiative Web site, http://teachers.yale.edu.
  41. For qualitative evidence of these benefits, see the Pittsburgh Teachers Institute Transcripts of Focus Groups with Elementary/Middle School Teachers and Principals, and High School Teachers and Principals; and the publications on the Yale-New Haven Teachers Institute and the National Demonstration Project available at the Yale National Initiative Web site, http://teachers.yale.edu.
  42. The multilevel design by Dr. Ellen Kisker is available as an online appendix to this report at http://teachers.yale.edu.
  43. The data set and the data analysis memos by David Seidenfeld on which this report is based can be found at http://teachers.yale.edu.
  44. The standardized tests employed in the study are the Connecticut Mastery Test (CMT), which was administered in the study period in grades 4, 6, and 8, and the Connecticut Academic Performance Test (CAPT), which was administered in grade 10. During the period of the study, the tests assessed student learning in Mathematics, Reading, and Writing (grades 4, 6, 8, and 10) and Science (grade 10).
  45. Elementary teachers, who were not initially eligible for New Haven seminars, came to be more proportionately represented over the years. During the test period, a number of New Haven elementary schools were made part of K-8 schools. Many teachers were reclassified as K-8 school teachers, leading to an apparent decline in elementary school teacher participation.
  46. Ingersoll, R., "Teacher Turnover and Teacher Shortages: An Organizational Analysis," American Educational Research Journal 38: 499-534 (2001).
  47. Kane et al., "What Does Certification Teach Us About Teacher Effectiveness? Evidence from New York City," 30.
  48. The technical research guide, designed by Dr. Ellen Kisker, is available as an online appendix to this report at http://teachers.yale.edu.