Evaluations and Results

Very important in the success of this effort has been the commitment to documentation, evaluation, and dissemination of results from the points of view of all participants. Throughout its history the Yale-New Haven Teachers Institute has arranged for, and learned from, both internal and external evaluations. These have been embodied in its Annual Reports and other publications. The National Demonstration Project and the Yale National Initiative have continued that process of multiple evaluation.

The National Demonstration Project included the annual administration of similar questionnaires at the sites establishing new Institutes and a one-time survey on unit use by participating and non-participating teachers at each site. These surveys were analyzed in a 2004 study sponsored by the Institute, "To Motivate My Students," prepared by Professor Rogers M. Smith of the University of Pennsylvania. It found that teachers at all the sites consistently rated the Institute programs higher than other professional development programs in developing the knowledge, skills, enthusiasm, high expectations of students, and capacities to motivate students that most studies indicate to be central to successful teaching. An external evaluator of the National Demonstration Project, Policy Studies Associates, found that the Project had succeeded in showing that the Institute model could be replicated "where districts value teachers' learning" and "intellectual rigor in the development of curriculum units." Using qualitative techniques including seminar and classroom observations, in-depth interviews, and focus groups, sociologists at the University of Houston and educators at Carnegie Mellon University and Chatham College, Pittsburgh, also conducted their own evaluations of their Teachers Institutes during the National Demonstration Project, with similarly positive results. The internal evaluations, based in part upon observations in site visits, the results of questionnaires, published curriculum units, and Annual Reports from participating Institutes, have been embodied in Annual Reports to the funding organizations. They have been supplemented by external evaluations of several kinds conducted by Policy Studies Associates, Cornerstone Evaluation Associates, and researchers from Carnegie Mellon University and the University of Houston.

These evaluations show that in all demonstration sites there were positive results similar to those obtained in New Haven over many years. Both Policy Studies Associates and Professor Smith concluded that the National Demonstration Project had "succeeded in reaching its goal" of replication of the Yale-New Haven model within a relatively short period of time in four sites that are considerably larger than New Haven. Smith noted that the new Institutes produced results that were remarkably similar to each other and to experiences in New Haven, and markedly better than those reported by most existing forms of professional development. These results occurred despite significant demographic differences among the cities.

The Teachers Institute Approach Promotes Teacher Quality

As Smith pointed out, recent research indicates that the single most important factor in student performance is teacher quality. The consensus of researchers and teachers is that many existing forms of professional development are cursory, dreary exercises that leave teachers bored and resentful, not informed or inspired. The approach of a Teachers Institute, however, significantly strengthens teachers in all five of the major dimensions of teacher quality: it helps to produce teachers who really know their subjects; who have good basic writing, mathematics and oral presentation skills; who expect their students to achieve; who are enthusiastic about teaching; and who can motivate all children to learn.

According to Smith's analysis, teachers in the new Institutes chose to participate out of desires to improve themselves in exactly these areas. In each city, teachers participated out of desires to obtain curriculum suited to their needs, to increase their mastery of their subjects, and especially to obtain materials to motivate their students. Ninety-five percent of all participating teachers rated the Institute seminars "moderately" or "greatly" useful. Similar percentages said the seminars increased their knowledge, improved their skills and morale, and raised their expectation of students. Smith also found that the Institutes served to foster teacher leadership, to develop supportive teacher networks, to heighten university faculty commitments to improving public education, and to foster more positive partnerships between school districts and institutions of higher education. After teaching their curriculum units, two thirds of all participants rated them superior to all other curricula they had used. Roughly sixty percent of all participants rated student motivation and attention as higher during these units, producing substantially greater content mastery. These curriculum units, as Smith noted, emphasized teacher-led discussion, writing exercises, and activities designed to strengthen speaking, listening, vocabulary, reasoning skills, and mathematics skills.

According to the report from Policy Studies Associates, there is "clear evidence of important accomplishments, reflected in the number of seminars provided in the Institutes, the number of Fellows who participated in these seminars, and the number of curriculum units the Fellows produced." It stated further:

Large majorities of Fellows were unequivocal in saying that their experience in the Institutes, especially the preparation of a curriculum unit, gave them a real sense of accomplishment and rekindled their excitement about learning. As one Fellow put it: "To be teachers, we must also be learners." When asked in interviews to compare their experience in the Institutes with their experience in other kinds of professional development, teachers agreed that the Institutes are vastly superior.

The report by Rogers M. Smith concluded:

No single program can overcome the enormous obstacles to educational achievement faced by economically disadvantaged students, usually from racial, ethnic, and linguistic minorities, in large American cities today. But if recent researchers are right to contend that the single most important factor in student achievement is teacher quality, and if quality teachers are indeed knowledgeable, skilled, and enthusiastic, with high expectations for their students and the means to motivate students to reach those expectations, then the National Demonstration Project provides strong evidence for the value of the Teachers Institute approach.

Report Links Institute Participation to Teacher Retention

A further report by Professor Smith, released in 2009, reaffirms and expands on his earlier findings. The report, To Strengthen Teaching: An Evaluation of Teachers Institute Experiences, presents results of a study of participants in Teachers Institutes in Houston, New Haven, Philadelphia, and Pittsburgh from 2003-2008 and in National Initiative seminars at Yale from 2005-2008. As before, Professor Smith concluded, "Teachers Institutes significantly strengthen teachers in all five of the major dimensions of teacher quality." After surveying the literature on teaching professional development, he added, "They also include all seven elements now recognized to be crucial in successful professional development programs." The study also examined retrospectively the results of Institute participation for New Haven teachers between 2000 and 2005. According to Smith:

The New Haven quantitative study indicates that Institute seminars attract a broad range of teachers from every observable demographic category and that those who choose to be Fellows are much more likely to continue teaching in the district than those who are not. The study also shows that Institute participants had nearly twice the retention rate of non-participants in local teaching. Because research suggests that experience within a district is more strongly associated with teaching effectiveness than earlier experiences elsewhere, this finding is especially notable.

Summarizing his findings in the 2009 report, Professor Smith said:

In Institute seminars teachers gain more sophisticated content knowledge and also enhance their skills as they prepare curriculum units adapting the themes of their seminars for their students. Most teachers are enthusiastic about the seminars and the opportunity to teach the units they have written. They expect more of the students taking them. And they succeed in motivating their students to learn at higher levels. The data on unit use also show that after teaching their Institute units two-thirds of all participants rated them superior to all other curriculum they had used. Roughly 60% of all participants rated student motivation and attention as higher during these units, producing substantially greater content mastery.... These data strongly support the conclusions that virtually all teachers who complete Institute seminars feel substantially strengthened in their mastery of content knowledge and their professional skills more generally, while they also develop higher standards for what their students can achieve.

As part of the Yale National Initiative, all existing and prospective Teachers Institutes and their school districts are being encouraged to assist in establishing data collection systems that will support ongoing evaluation of the Teachers Institutes on student and teacher achievements. The 2009 report describes some of the possibilities that exist for further evaluation and for investigation into the relationship between Institute participation and teacher effectiveness.

Evaluations:

"To Strengthen Teaching: An Evaluation of Teachers Institute Experiences"
"To Motivate My Students: An Evaluation of the National Demonstration Project of the Yale-New Haven Teachers Institute"
"Pittsburgh Teachers Institute - Lessons Learned"