Here is a good summary of what the two sides are fighting over.
A lot of the disagreement involves the use of value added measures based on student test scores to evaluate teachers. I will try to post more on this soon, since I do a lot of research on this stuff, but I am swamped with PhD work.
Monday, September 10, 2012
Thursday, August 2, 2012
Algebra Problems
His main point is that algebra just isn't that useful of a skill for most workers, so why learn it? A main function of schooling is to raise the productivity of our labor force, so why spend valuable classroom time on things that don't do so?
The other issue is that lots of people fail at the subject. Failing could lead them to drop out of school. Also, if student's don't pass then they may be ineligible for things like admissions to higher education programs.
I am not convinced that algebra is so useless for most people. It's intuitive to me that learning algebra has an effect on a person's higher order thinking skills, which may lead to better later life outcomes. Also, struggling through something difficult can be a valuable thing for a person.
However, suppose that algebra really adds nothing to a worker's productivity, and it's causing students to flunk out of school. It can still serve a valuable economic function.
The reason has to do with signaling. A person's performance in algebra is likely related to lots of characteristics that are difficult to measure like intelligence, motivation, attitudes, etc. All of these things affect the ability of students to finish school and the productivity of workers. You can never say that a particular person is less able etc. because they didn't finish algebra, but on average they are.
It's costly to educate students. A lot of resources go into admissions, administration, student support, and of course instruction, so it's important that students are able and motivated enough to finish their coursework. It seems perfectly reasonable to me that universities set high bars for mathematics achievement, as the author mentions, even if it has no bearing on the curriculum. Not all students can make it in every program, and avoiding bad matches, where the student is unable to finish their degree, can be useful for both parties. I wish they could, but that isn't the world we live in.
The same argument applies for the job market. Employers spend a lot of time and money on job search and so do employees. It's important that when an employer hires someone the match is a good one. Otherwise, the two have to separate, and the game starts all over again.
If everyone breezes through high school or college without a challenge then a degree loses its signaling power. Signals can help avoid bad matches or help speed up attrition when there is a good chance of that happening anyway.
I still think the main benefits from algebra are that it actually does raise your productivity as a worker, but we should be careful when we consider stripping the curriculum of material that is challenging and a stumbling block, since in some ways that's a feature and not a bug.
Wednesday, July 25, 2012
Teacher Merit Pay Might Raise Student Achievement After All
A few weeks ago, I wrote a post on whether teacher merit pay is effective at increasing student achievement. I felt that the evidence was pretty weak for the whole idea.
Lo and behold, a new paper came out recently on just this subject. The authors are Roland Fryer, Steven Levitt (of Freakonomics fame), John List, and Sally Sadoff. Here is a link, although it's gated.
They conduct a randomized controlled trial in which teachers are randomly assigned to a bonus program, which gives money for strong gains in student achievement. The program is fairly similar to the Nashville study that I discussed before.
The really interesting thing is the way the bonuses are given out. Some of the teachers are paid $4,000 at the beginning of the school year, before any teaching has taken place. Another group are given the bonus at the end of the year, and the amount depends on how well their students perform. For the teachers paid at the beginning of the year, if their students don't perform well, then they are forced to give potentially all of the money back.
The notion that people might be more motivated to not lose something than gain something has a long literature dating back to some work by Tversky and Kahneman (another person with some interesting books).
The teachers are randomly assigned to the groups. It turns out that the effect of the program on student achievement is weak and mostly statistically insignificant for teachers given money at the end of the year, as it was in the Nashville study. However, for the group of teachers that faced the prospect of giving back part of the $4,000, the effect was pretty big! Students with teachers that faced the loss had test scores that were .201 to .398 standard deviations higher on average compared to a control group of students. A .398 standard deviation improvement is similar to going from a 500 on the SAT to a 544, and that is for every student in the class.
I would like some more information about the randomization, like whether principals had the opportunity to assign the program teachers to better or worse classes. However, I think this paper is really interesting, and it's hard for me to come up with an alternate explanation for why the upfront bonus that can be taken away is so much more effective.
I was skeptic of teacher merit pay, but when new evidence comes along we are supposed to change our minds. Maybe teacher merit pay can work if it's done the right way.
Lo and behold, a new paper came out recently on just this subject. The authors are Roland Fryer, Steven Levitt (of Freakonomics fame), John List, and Sally Sadoff. Here is a link, although it's gated.
They conduct a randomized controlled trial in which teachers are randomly assigned to a bonus program, which gives money for strong gains in student achievement. The program is fairly similar to the Nashville study that I discussed before.
The really interesting thing is the way the bonuses are given out. Some of the teachers are paid $4,000 at the beginning of the school year, before any teaching has taken place. Another group are given the bonus at the end of the year, and the amount depends on how well their students perform. For the teachers paid at the beginning of the year, if their students don't perform well, then they are forced to give potentially all of the money back.
The notion that people might be more motivated to not lose something than gain something has a long literature dating back to some work by Tversky and Kahneman (another person with some interesting books).
The teachers are randomly assigned to the groups. It turns out that the effect of the program on student achievement is weak and mostly statistically insignificant for teachers given money at the end of the year, as it was in the Nashville study. However, for the group of teachers that faced the prospect of giving back part of the $4,000, the effect was pretty big! Students with teachers that faced the loss had test scores that were .201 to .398 standard deviations higher on average compared to a control group of students. A .398 standard deviation improvement is similar to going from a 500 on the SAT to a 544, and that is for every student in the class.
I would like some more information about the randomization, like whether principals had the opportunity to assign the program teachers to better or worse classes. However, I think this paper is really interesting, and it's hard for me to come up with an alternate explanation for why the upfront bonus that can be taken away is so much more effective.
I was skeptic of teacher merit pay, but when new evidence comes along we are supposed to change our minds. Maybe teacher merit pay can work if it's done the right way.
Tuesday, July 10, 2012
Embarrassing Fact of the Day
Felix Salmon has an article on how economists get tripped up by statistics. See if you can solve the problem.
Apparently 72% of professional economists at leading universities not only got the question wrong, but got it extremely wrong when the information was presented in a regression output table.
I don't know what to say. This makes me sad.
Update: I decided to post my solution in the comments section for anyone interested.
Apparently 72% of professional economists at leading universities not only got the question wrong, but got it extremely wrong when the information was presented in a regression output table.
I don't know what to say. This makes me sad.
Update: I decided to post my solution in the comments section for anyone interested.
Monday, June 25, 2012
Age Ain't Nothing But a Number
There was an interesting article in Sunday New York Times on political and economic differences between generations.
The article's main focus is on differences in voter preferences and how a large chunk of federal spending is a transfer from young people to old people (Social Security, Medicare, etc).
Their are also some really cool graphs on changes from 1992-2010 in median net wealth and income for different age groups. It's pretty well known that median income growth has been pretty stagnant for at least the past decade, and this is apparent in the data. However, there are some large differences across age groups, with older people doing relatively well.
I wasn't aware of this, and I don't have a great explanation for it. It might have something to do with older people earning a larger fraction of their income in capital gains, but I don't have a great story.
Update: I have gotten some very good comments on this post. Thanks for that!
I was interested in knowing what the picture looked like before the recession. The recession struck around the end of 2007 and most of the damage occurred in 2008. Here are the pictures for change in net worth and income substituting 2007 in for 2010, which hopefully shows us what was happening prior to the recession.
It actually looks quite different from the NYT charts. It seems that all age groups were seeing gains prior to the recession, and that the recession has taken a different toll across these age groups.
Those age 65 and older have held onto their gains through the recession, and just about everyone else has taken a step back. I agree with Mark in the comments that this likely has to do with retirees having more guaranteed income and less risky investments. Also, his theory on housing values makes sense to me.
By the way, the entire data series for net worth can be found here:
http://www.federalreserve.gov/
in this file:
Estimates inflation-adjusted to 2010 dollars (5.5 MB Excel)
And for net income here:
http://www.census.gov/hhes/
in table H-10:
The article's main focus is on differences in voter preferences and how a large chunk of federal spending is a transfer from young people to old people (Social Security, Medicare, etc).
Their are also some really cool graphs on changes from 1992-2010 in median net wealth and income for different age groups. It's pretty well known that median income growth has been pretty stagnant for at least the past decade, and this is apparent in the data. However, there are some large differences across age groups, with older people doing relatively well.
I wasn't aware of this, and I don't have a great explanation for it. It might have something to do with older people earning a larger fraction of their income in capital gains, but I don't have a great story.
Update: I have gotten some very good comments on this post. Thanks for that!
I was interested in knowing what the picture looked like before the recession. The recession struck around the end of 2007 and most of the damage occurred in 2008. Here are the pictures for change in net worth and income substituting 2007 in for 2010, which hopefully shows us what was happening prior to the recession.
It actually looks quite different from the NYT charts. It seems that all age groups were seeing gains prior to the recession, and that the recession has taken a different toll across these age groups.
Those age 65 and older have held onto their gains through the recession, and just about everyone else has taken a step back. I agree with Mark in the comments that this likely has to do with retirees having more guaranteed income and less risky investments. Also, his theory on housing values makes sense to me.
By the way, the entire data series for net worth can be found here:
http://www.federalreserve.gov/
in this file:
Estimates inflation-adjusted to 2010 dollars (5.5 MB Excel)
And for net income here:
http://www.census.gov/hhes/
in table H-10:
Saturday, June 2, 2012
Will teacher merit pay raise student achievement?
There has been a push recently to give teachers merit pay for increasing student test scores. In fact, I think both presidential candidates support it.
There is some pretty nice theory supporting the idea of merit pay. There is also some pretty weak evidence that it actually works.
Merit pay could work in two ways.
The first is that it will change who chooses to teach. If we can pay all of the best teachers well and the lousy teachers poorly, then over time hopefully we can get the lousy teachers to do something else. The idea is to change the makeup of the teaching profession, hopefully by washing out the poor teachers by not paying them very much.
I am not totally convinced that merit pay will do this. The measures of how much knowledge a teacher has added to their students are far from perfect. In fact, teachers rated highly in one year may be rated in the bottom the next year using these measures (see Aaronson et al (2007), McCaffrey et al (2009), and Stacy (20??) as soon as I finish it). Anyway, in a merit pay system, a teachers salary might swing a lot from year to year. People tend not to like that, so it may be that the best teachers, who also have the best outside opportunities, decide to leave teaching altogether. This is all conjecture though.
How merit pay affects the teaching profession in the long term is an empirical question. Unfortunately, I don't think anyone has answered it convincingly. We don't know, basically.
The other way merit pay can work is to motivate teachers already in the profession. This notion has a big economics literature. It's called moral hazard. Society hires teachers to teach their children. The teachers don't (so the argument goes) have as much a stake in the outcomes of the students as their parents. Without any sort of tie of salary to performance, teachers might be lazy and not work hard. Merit pay gives teachers an incentive to work harder, and hopefully this will lead to more effort and better student outcomes.
We do have some good evidence on this idea. In a study ran from 2006-2009, teachers in Nashville, TN were paid bonuses of up to $15,000 for improving student test scores. This was a randomized experiment, which despite what some people say, is the gold standard of empirical research. You can read through the report here. It's not perfect, but it's probably as good as we can do.
Basically they found zero effect of the merit pay bonus system on student achievement. This was a short run study, so it doesn't offer us much insight into the first argument. However, this is a big blow to the second argument. The authors of the report were pretty careful, and it's hard to come up with a good reason for why the chance for a $15,000 bonus didn't motivate people if they could be.
There is a great sentence in the conclusion of the report:
I am not convinced that merit pay works. Maybe it can with the right design, but I don't know. I don't think there is much data supporting the idea that we can improve student achievement with teacher merit pay. There is strong evidence against the second argument, and there is little or no evidence either way for the first argument.
Also, if anyone has any papers that contradict these thoughts, then let me know.
There is some pretty nice theory supporting the idea of merit pay. There is also some pretty weak evidence that it actually works.
Merit pay could work in two ways.
The first is that it will change who chooses to teach. If we can pay all of the best teachers well and the lousy teachers poorly, then over time hopefully we can get the lousy teachers to do something else. The idea is to change the makeup of the teaching profession, hopefully by washing out the poor teachers by not paying them very much.
I am not totally convinced that merit pay will do this. The measures of how much knowledge a teacher has added to their students are far from perfect. In fact, teachers rated highly in one year may be rated in the bottom the next year using these measures (see Aaronson et al (2007), McCaffrey et al (2009), and Stacy (20??) as soon as I finish it). Anyway, in a merit pay system, a teachers salary might swing a lot from year to year. People tend not to like that, so it may be that the best teachers, who also have the best outside opportunities, decide to leave teaching altogether. This is all conjecture though.
How merit pay affects the teaching profession in the long term is an empirical question. Unfortunately, I don't think anyone has answered it convincingly. We don't know, basically.
The other way merit pay can work is to motivate teachers already in the profession. This notion has a big economics literature. It's called moral hazard. Society hires teachers to teach their children. The teachers don't (so the argument goes) have as much a stake in the outcomes of the students as their parents. Without any sort of tie of salary to performance, teachers might be lazy and not work hard. Merit pay gives teachers an incentive to work harder, and hopefully this will lead to more effort and better student outcomes.
We do have some good evidence on this idea. In a study ran from 2006-2009, teachers in Nashville, TN were paid bonuses of up to $15,000 for improving student test scores. This was a randomized experiment, which despite what some people say, is the gold standard of empirical research. You can read through the report here. It's not perfect, but it's probably as good as we can do.
Basically they found zero effect of the merit pay bonus system on student achievement. This was a short run study, so it doesn't offer us much insight into the first argument. However, this is a big blow to the second argument. The authors of the report were pretty careful, and it's hard to come up with a good reason for why the chance for a $15,000 bonus didn't motivate people if they could be.
There is a great sentence in the conclusion of the report:
In sum, the introduction of performance incentives in MNPS middle schools did not set off sigificant negative reactions of the kind that have attended the introduction of merit pay elsewhere. But neither did it yield consistent and lasting gains in test scores. It simply did not do much of anything.As far as I could tell, the authors best explanation for why their was little effect was that teachers were mostly working pretty hard to begin with. They surveyed the teachers, and 80% reported that they were teaching as effectively as they could even before the program. If that's the case, then we should only expect a marginal effect.
I am not convinced that merit pay works. Maybe it can with the right design, but I don't know. I don't think there is much data supporting the idea that we can improve student achievement with teacher merit pay. There is strong evidence against the second argument, and there is little or no evidence either way for the first argument.
Also, if anyone has any papers that contradict these thoughts, then let me know.
Thursday, May 31, 2012
What would happen if we increase teacher wages from $65,000 to $150,000?
In today's Washington Post, there is an article on teacher unions (click on link to go to article) and how they operate differently in other countries. The article mentions that many nations with the highest student achievement ( Finland, Singapore, South Korea) are heavily unionized, so it may be the case that teacher unions don't necessarily cripple an education system.
The author then proposes an interesting idea, which is to increase teacher starting pay from an average of $65,000 to $150,000. I think the author believes that this will somehow change the relationship between teacher unions and the educational system. I don't know. Maybe it will, and maybe it won't.
Anyway, I thought it would be interesting to look at the evidence for increasing teacher wages on student achievement.
The basic argument for why increasing salaries will increase student achievement is that with higher salaries school districts will be able to attract (on average) higher quality people into the teaching profession. Better teachers will lead to better student outcomes.
This is an economics blog, so here is a diagram.
There are two papers that I jumped to when thinking about this question. The first is a 1995 JHR paper by Ballou & Podgursky. The other is a 2000 REStat paper by Loeb & Page. Both are a little old, so if someone has some good recent papers, let me know.
Ballou & Podgursky model three things: the decision to train for a teaching career in college, the decision to seek a teaching career after college, and the decision to continue teaching each year. Their insight is that increasing salaries for teachers keeps teachers already in the profession in for longer. This can actually crowd out new teachers from entering the profession. This is bad if you want to increase overall teacher quality in the profession, since the old (bad?) teachers stick around for longer. The other insight is that it can take a long time (decades?) for the old to be replaced by the new better teachers, and that the size of the change depends on how well school districts are able or want to pick the best talent.
Loeb & Page look empirically at what happens when salaries increase for teachers. They use state-level panel data and look at teacher wages relative to other professions teachers could enter. They find that increasing teacher salaries by 10% today reduces high school dropout rates somewhere between 3% and 6% (percent not percentage points) 10 years later.
This gets us closer to the original question. It's hard to say exactly what effect going from $65,000 to $150,000 (we are talking about a 131% increase not a 10% increase) will have on achievement. You can't take that 3-6% number too literally, since at some point the model would predict a negative dropout rate, which is impossible, but the point is that increasing teacher salaries to attract better talent does seem to have an effect.
To conclude, I am not exactly sure how big of an effect going from salaries of $65,000 to $150,000 would produce. It would probably be a pretty large effect, but it might take a long time to reap the fruits.
The author then proposes an interesting idea, which is to increase teacher starting pay from an average of $65,000 to $150,000. I think the author believes that this will somehow change the relationship between teacher unions and the educational system. I don't know. Maybe it will, and maybe it won't.
Anyway, I thought it would be interesting to look at the evidence for increasing teacher wages on student achievement.
The basic argument for why increasing salaries will increase student achievement is that with higher salaries school districts will be able to attract (on average) higher quality people into the teaching profession. Better teachers will lead to better student outcomes.
This is an economics blog, so here is a diagram.
There are two papers that I jumped to when thinking about this question. The first is a 1995 JHR paper by Ballou & Podgursky. The other is a 2000 REStat paper by Loeb & Page. Both are a little old, so if someone has some good recent papers, let me know.
Ballou & Podgursky model three things: the decision to train for a teaching career in college, the decision to seek a teaching career after college, and the decision to continue teaching each year. Their insight is that increasing salaries for teachers keeps teachers already in the profession in for longer. This can actually crowd out new teachers from entering the profession. This is bad if you want to increase overall teacher quality in the profession, since the old (bad?) teachers stick around for longer. The other insight is that it can take a long time (decades?) for the old to be replaced by the new better teachers, and that the size of the change depends on how well school districts are able or want to pick the best talent.
Loeb & Page look empirically at what happens when salaries increase for teachers. They use state-level panel data and look at teacher wages relative to other professions teachers could enter. They find that increasing teacher salaries by 10% today reduces high school dropout rates somewhere between 3% and 6% (percent not percentage points) 10 years later.
This gets us closer to the original question. It's hard to say exactly what effect going from $65,000 to $150,000 (we are talking about a 131% increase not a 10% increase) will have on achievement. You can't take that 3-6% number too literally, since at some point the model would predict a negative dropout rate, which is impossible, but the point is that increasing teacher salaries to attract better talent does seem to have an effect.
To conclude, I am not exactly sure how big of an effect going from salaries of $65,000 to $150,000 would produce. It would probably be a pretty large effect, but it might take a long time to reap the fruits.
Subscribe to:
Posts (Atom)