I said I was going to write about leadership and management books this week, but something more interesting intervened. I spent some time in the Teach for America offices, and it really got me thinking.
As most of you know, TFA has been remarkably successful doing what many in education predicted was impossible: placing graduates of the country’s most prestigious universities in our highest-need classrooms. TFA’s ambition, drive, and discipline are legendary, and their results speak for themselves. In 1990, TFA started with a corps of 500. Since then, 17,000 have joined TFA nationwide. About 1,000 TFA teachers are currently teaching at the DOE, and many TFA alumni have moved into DOE leadership roles. (In fact, sixty-five percent of TFA alumni in the New York region stay in education.) TFA continues to grow like crazy, and to attract far more applicants than it can accept. In 2006, 10 percent of the graduating classes from Amherst and Yale, 7 percent from Caltech, and 6 percent from Columbia, Northwestern and Rice applied to TFA. Outcome-based research studies differ to some extent, but on balance, I’m willing to conclude that TFA teachers are on average at least as effective as traditionally prepared teachers, and some studies have found them to be more effective, especially in math.
I think there is a lot to learn from TFA’s extraordinary organizational accomplishments in the face of obstacles many thought were insurmountable. It looks to me like they excel in three important domains:
They are relentlessly clear about what they believe. TFA proclaims that the achievement gap in this country is shameful and intolerable; that it cannot be blamed on poor children and their families; and that it can and must be changed. Here’s an excerpt from the TFA website:
“Each year, a Gallup poll asks the general public why we have low educational outcomes in low-income communities. Out of 20 options, the public answers that the top factors are lack of student motivation, lack of parental involvement, and home-life issues. In answer to the same question with the same 20 options, our corps members respond at the end of their two-year teaching commitments that the top factors are teacher quality, school leadership, and expectations of students. Based on their experiences working with students and families, corps members believe we have an achievement gap in our country not because kids lack motivation or families don't care, but rather because of the choices we're making as a society…..Teach For America alumni are working from all sectors to challenge the prevailing notions about educational inequity, and in turn to inform our priorities, policies, and practices in important ways.”
My bet is you wouldn’t hang around TFA long if you were willing to justify the status quo as out of educators’ control.
The culture is built around taking personal responsibility. This is clear from the TFA core values, which are prominently posted on the New York office wall, visible to employees, applicants, and visitors the minute they step off the elevator:
Relentless pursuit of results - We assume personal responsibility for achieving ambitious, measurable results in pursuit of our vision. We persevere in the face of challenges, seek resources to ensure the best outcomes, and work towards our goals with a sense of purpose and urgency.
Sense of possibility- We approach our work with optimism, think boldly, and greet new ideas openly.
Disciplined thought - We think critically and strategically in search of the best answers and approaches, reflect on past experiences and data to draw lessons for the future, and make choices that are deeply rooted in our mission.
Respect and humility- We value all who are engaged in this challenging work. We keep in mind the limitations of our own experiences and actively seek out diverse perspectives.
Integrity- We ensure alignment between our actions and our beliefs, engage in honest self-scrutiny, and do what's right for the broader good.
Imagine if you wrote an honest description of what the public education bureaucracy is seen as valuing in these domains. How would it contrast to the TFA values?
They live a concrete vision of service to others. This one gets at a key distinction we work on every day in DHR. TFA has made itself an important force in an entrenched national educational system that is known for being hostile to outsiders, and in which few other change agents have gained similar traction. I think that TFA’s success has something to do with the fact that they view educators as valued clients, whose trust and business they must earn. Nobody forces school systems to work with TFA. TFA listens closely to the needs of the schools, principals, and superintendents that they serve, and they design processes and structures that work for their clients, not that are convenient for them. To put it in “DOE terms”, it’s a culture built for performance, not compliance. The leaders at TFA know that they will not accomplish their larger goal of serving kids if they cannot offer great service to the schools they work with.
Of course, it’s a lot easier to build a culture when you have to earn your clients from Day 1….you get it right or the organization ceases to exist. But it’s worth asking: what would happen if schools got to choose whether to do business with DHR? As we redesign HR processes, make the hard transitions to bring HRConnect to fruition, support reorganizations, and continue the work of transforming DHR, I hope we keep asking: what would happen if we had to compete for the right to serve kids? Would we earn it?
03-22-07 More on Educational Research
This Weekly Reader is about the contribution of economists to educational research—arguably the geekiest Weekly Reader topic yet. I realized I might not have ever said explicitly why I think those of us who work in Human Resources should be curious and literate about educational research. I see our job as helping the system find, develop, serve, and nurture a workforce that can cause kids to learn. So the more information we can get about the characteristics and behaviors that cause learning, the better able we’ll be to do our jobs. The link is obvious in the case of teachers. The more we know about what makes teachers successful in the classroom, the better we can help in selecting and training them, and the better we can serve their needs as employees. But it doesn’t stop there. I know a system that, driven by some research findings about kids’ readiness to learn, launched a new approach to hiring, training, and supporting janitors and school bus drivers, because they saw them as people who have important impacts on kids’ attitudes. There’s a whole research field devoted to adult learning theory, which can help us understand what causes our employees to succeed in their own learning challenges. Research also helps us understand how and where limited resources can be put to best use.
So now a little bit about economists. More and more, influential educational research studies seem to be based on econometric models. These models range from simple linear regression, in which we look at the relationship between x and y, plotted as a line (for every x increase in years’ education, there is a y increase in income, for example), to very complex models that can estimate the relationships between many different types of variables. These models allow researchers to look at a whole bunch of factors that might influence gains in student learning, and try to see the impact size and relative importance of each factor. They can pose questions like: “holding students’ socioeconomic status constant, what’s the relative effect of teacher certification?” These kinds of models aren’t unique to economists—they are also the tools of political scientists, psychologists, and sociologists, to name a few. But over the past decade or so, economists seem to have taken the lead in quantitative research in education. The Spencer Foundation, a blue-chip organization that supports research to improve schools, explained the kinds of questions economists ask in this way:
“First, economists tend to focus on costs as well as benefits. They ask, for example, not only whether reducing class size results in improved student learning, but also whether the gains relative to the costs compare favorably with other ways to use scarce resources. In other words, said one participant, "we should move beyond asking ‘does reducing class size help?’ and ask instead ‘how much does it help?’" Second, economics provides a strong theoretical framework, one that is particularly valuable in analyzing the operation of markets. The theory provides guidance about the types of data that are needed to evaluate the consequences of educational policies that concern interventions in markets, such as pay increases for teachers, and educational vouchers for students. Third, economists tend to value long-term as well as short-term outcomes. This is critical to understanding the consequences of many programs. . . . Finally, economists have solid statistical knowledge, particularly in non-experimental statistics. "We know . . . about how to handle situations where you don’t have all the data you want and there are omitted variables in your explanatory equation; that’s one of our strengths. . . .”
For more on the educational research approaches of economists, go to:
http://www.spencer.org/publications/conferences/Economics/report.htm
In a prior Weekly Reader, I talked about the contributions to our understanding of teacher quality at the DOE that have been made by Tom Kane of Harvard, Jonah Rockoff of Columbia, and Douglas Staiger of Dartmouth, all trained economists. If you missed it, you can see this (and all prior Weekly Readers, by the way) at:
http://leighsweeklyreader.blogspot.com/2007_01_01_archive.html (scroll down to the 12-05-06 edition)
And if econometric models appeal to your inner geek, here are a couple other guys who have done very interesting work that has made a mark on the national stage:
http://edpro.stanford.edu/hanushek/content.asp?ContentId=61
http://evans.washington.edu/fac/Goldhaber/
Next week, some of my favorite books on leadership and organizational management, and why I think they are relevant to our challenges in education.
So now a little bit about economists. More and more, influential educational research studies seem to be based on econometric models. These models range from simple linear regression, in which we look at the relationship between x and y, plotted as a line (for every x increase in years’ education, there is a y increase in income, for example), to very complex models that can estimate the relationships between many different types of variables. These models allow researchers to look at a whole bunch of factors that might influence gains in student learning, and try to see the impact size and relative importance of each factor. They can pose questions like: “holding students’ socioeconomic status constant, what’s the relative effect of teacher certification?” These kinds of models aren’t unique to economists—they are also the tools of political scientists, psychologists, and sociologists, to name a few. But over the past decade or so, economists seem to have taken the lead in quantitative research in education. The Spencer Foundation, a blue-chip organization that supports research to improve schools, explained the kinds of questions economists ask in this way:
“First, economists tend to focus on costs as well as benefits. They ask, for example, not only whether reducing class size results in improved student learning, but also whether the gains relative to the costs compare favorably with other ways to use scarce resources. In other words, said one participant, "we should move beyond asking ‘does reducing class size help?’ and ask instead ‘how much does it help?’" Second, economics provides a strong theoretical framework, one that is particularly valuable in analyzing the operation of markets. The theory provides guidance about the types of data that are needed to evaluate the consequences of educational policies that concern interventions in markets, such as pay increases for teachers, and educational vouchers for students. Third, economists tend to value long-term as well as short-term outcomes. This is critical to understanding the consequences of many programs. . . . Finally, economists have solid statistical knowledge, particularly in non-experimental statistics. "We know . . . about how to handle situations where you don’t have all the data you want and there are omitted variables in your explanatory equation; that’s one of our strengths. . . .”
For more on the educational research approaches of economists, go to:
http://www.spencer.org/publications/conferences/Economics/report.htm
In a prior Weekly Reader, I talked about the contributions to our understanding of teacher quality at the DOE that have been made by Tom Kane of Harvard, Jonah Rockoff of Columbia, and Douglas Staiger of Dartmouth, all trained economists. If you missed it, you can see this (and all prior Weekly Readers, by the way) at:
http://leighsweeklyreader.blogspot.com/2007_01_01_archive.html (scroll down to the 12-05-06 edition)
And if econometric models appeal to your inner geek, here are a couple other guys who have done very interesting work that has made a mark on the national stage:
http://edpro.stanford.edu/hanushek/content.asp?ContentId=61
http://evans.washington.edu/fac/Goldhaber/
Next week, some of my favorite books on leadership and organizational management, and why I think they are relevant to our challenges in education.
03-07-07 Research Methods
Conversations about education are often peppered with the phrase “research shows……” But not all research is alike, and I thought it might be useful to provide a brief overview of educational research methods as I see them.
Educational research studies use both quantitative and qualitative methods. Quantitative research relies on statistical analysis to look for relationships, based on samples that are of sufficient size to produce meaningful results. For example, there are several quantitative studies that have shown a relationship between the SAT or ACT scores of teachers and the reading or math scores of the elementary students that they teach. The higher the teacher’s score, the higher the average student score. Since we know that there are important factors other than teacher scores that affect student scores (such as students’ socioeconomic status and students’ prior achievement, for example) studies like these add those variables into the equation in order to control for them. As my former statistics professor used to say, it’s the beauty of mathematics.
The advantage of quantitative research is that it relies on large samples to discern relationships that hold true in general. Thus, quantitative research can be a good basis for making policy decisions. We might decide, for example, that SAT score would be one factor to take into account in screening applicants for teacher positions. There are a few caveats to bear in mind when thinking about quantitative research, however. First, most researchers will only draw conclusions from results that are statistically significant at the .95 confidence level. That means that there is a less than 5 percent chance that the relationship shown is caused by chance. Five percent is small, but it’s not zero---lots of things happen that have less than a 5 percent probability. Thus, we usually would want to see a number of sound, large-scale studies showing the same relationship before we would make a policy decision. Second, we have to beware of spurious correlations. We can show statistically that there is a relationship between students’ shoe size and intelligence as measured by some tests. But we would be nuts to choose kids with big feet for gifted classes: kids will test higher as they age; their feet just come along for the ride. Finally, what’s true in general doesn’t hold true in every case. It’s important to keep the exceptions in mind and avoid making policy that harms those who don’t fit into the general pattern that the research shows. You have to keep your wits about you as you draw conclusions from, and make policy based on, quantitative studies.
While quantitative research usually draws its strength from large numbers, qualitative research is based on close observation and analysis. Sometimes a case study method is used, in which one or more situations of interest are studied in depth in order to try to understand the mechanisms at work. For example, a teacher quality study might focus on what teachers who are rated highly by their principals do in the classroom, in order to draw generalizable conclusions about the methods and behaviors that characterize good teaching. Qualitative research is particularly useful at teasing out answers to the “why” and “how” questions that quantitative research often can’t answer. Once we suspect, for example, that teachers with higher SAT scores might cause higher student achievement, a good follow-up qualitative study might videotape a group of high SAT teachers, and another group of lower SAT teachers, and try to determine how they differ. Maybe teachers with higher SAT scores understand math better. Or maybe they don’t give up in the face of obstacles, or have better self discipline, or set higher expectations. Understanding that would tell us a lot.
Qualitative studies add insight, richness and depth. What we have to bear in mind about qualitative studies, however, is that they are often based on just a few situations, and that observation is fallible. Some of the most effective teachers might not fit a particular researcher’s model of what good teaching should look like. (This is a particular concern of mine, I confess, since some of the people I’ve learned the most from in my life didn’t fit the typical education school picture of good teaching. And when I ask people to describe the teacher who made the most difference for them, I hear about a remarkable range of approaches and styles, some of which are pretty far out of the mainstream mold.) So you can see that I test research approaches against my own experience and observations, too, which some researchers decry, but seems to me essential and anyway unavoidable. (Sociology majors might remember the insights of C. Wright Mills, who drew on the private problems of the individual to help frame the big problems of social science research.)
In some schools of education, the qualitative and the quantitative researchers don’t mix. It’s a classic liberal/conservative standoff in which the qualitative researchers seem to believe that the quantitative researchers have no hearts, while the quantitative researchers seem to believe that the qualitative researchers have no brains. I think both types of research are important. Even the best tools can be misused, and research tools are no different. The key is to be clear about the benefits and limitations of all our research tools, and to remember that we are dealing with the complex lives of real kids in real schools.
I’m on vacation next week, but the following week I’ll provide a little summary of the current state of research on student achievement, and the growing contributions of the economists who are taking quantitative research to a new level.
Educational research studies use both quantitative and qualitative methods. Quantitative research relies on statistical analysis to look for relationships, based on samples that are of sufficient size to produce meaningful results. For example, there are several quantitative studies that have shown a relationship between the SAT or ACT scores of teachers and the reading or math scores of the elementary students that they teach. The higher the teacher’s score, the higher the average student score. Since we know that there are important factors other than teacher scores that affect student scores (such as students’ socioeconomic status and students’ prior achievement, for example) studies like these add those variables into the equation in order to control for them. As my former statistics professor used to say, it’s the beauty of mathematics.
The advantage of quantitative research is that it relies on large samples to discern relationships that hold true in general. Thus, quantitative research can be a good basis for making policy decisions. We might decide, for example, that SAT score would be one factor to take into account in screening applicants for teacher positions. There are a few caveats to bear in mind when thinking about quantitative research, however. First, most researchers will only draw conclusions from results that are statistically significant at the .95 confidence level. That means that there is a less than 5 percent chance that the relationship shown is caused by chance. Five percent is small, but it’s not zero---lots of things happen that have less than a 5 percent probability. Thus, we usually would want to see a number of sound, large-scale studies showing the same relationship before we would make a policy decision. Second, we have to beware of spurious correlations. We can show statistically that there is a relationship between students’ shoe size and intelligence as measured by some tests. But we would be nuts to choose kids with big feet for gifted classes: kids will test higher as they age; their feet just come along for the ride. Finally, what’s true in general doesn’t hold true in every case. It’s important to keep the exceptions in mind and avoid making policy that harms those who don’t fit into the general pattern that the research shows. You have to keep your wits about you as you draw conclusions from, and make policy based on, quantitative studies.
While quantitative research usually draws its strength from large numbers, qualitative research is based on close observation and analysis. Sometimes a case study method is used, in which one or more situations of interest are studied in depth in order to try to understand the mechanisms at work. For example, a teacher quality study might focus on what teachers who are rated highly by their principals do in the classroom, in order to draw generalizable conclusions about the methods and behaviors that characterize good teaching. Qualitative research is particularly useful at teasing out answers to the “why” and “how” questions that quantitative research often can’t answer. Once we suspect, for example, that teachers with higher SAT scores might cause higher student achievement, a good follow-up qualitative study might videotape a group of high SAT teachers, and another group of lower SAT teachers, and try to determine how they differ. Maybe teachers with higher SAT scores understand math better. Or maybe they don’t give up in the face of obstacles, or have better self discipline, or set higher expectations. Understanding that would tell us a lot.
Qualitative studies add insight, richness and depth. What we have to bear in mind about qualitative studies, however, is that they are often based on just a few situations, and that observation is fallible. Some of the most effective teachers might not fit a particular researcher’s model of what good teaching should look like. (This is a particular concern of mine, I confess, since some of the people I’ve learned the most from in my life didn’t fit the typical education school picture of good teaching. And when I ask people to describe the teacher who made the most difference for them, I hear about a remarkable range of approaches and styles, some of which are pretty far out of the mainstream mold.) So you can see that I test research approaches against my own experience and observations, too, which some researchers decry, but seems to me essential and anyway unavoidable. (Sociology majors might remember the insights of C. Wright Mills, who drew on the private problems of the individual to help frame the big problems of social science research.)
In some schools of education, the qualitative and the quantitative researchers don’t mix. It’s a classic liberal/conservative standoff in which the qualitative researchers seem to believe that the quantitative researchers have no hearts, while the quantitative researchers seem to believe that the qualitative researchers have no brains. I think both types of research are important. Even the best tools can be misused, and research tools are no different. The key is to be clear about the benefits and limitations of all our research tools, and to remember that we are dealing with the complex lives of real kids in real schools.
I’m on vacation next week, but the following week I’ll provide a little summary of the current state of research on student achievement, and the growing contributions of the economists who are taking quantitative research to a new level.
Subscribe to:
Posts (Atom)