Summer jobs programs — whether publicly funded or merely subsidized — have long been used to provide low-income youth with work experience and a paycheck, and to help keep them out of trouble while school is out of session. But do such initiatives actually achieve those goals?

Judd B. Kessler, a Wharton professor of business economics and public policy, along with Alexander Gelber, a fellow at the National Bureau of Economic Research and a professor at the University of California-Berkeley, and Adam Isen of the U.S. Treasury Department, examine the outcomes for participants in New York City’s summer youth employment program in the paper, “The Effects of Youth Employment: Evidence from New York City Summer Youth Employment Program Lotteries.” The results of their research were recently published in the Quarterly Journal of Economics. In an interview with Knowledge at Wharton, Kessler explains what they learned.

An edited transcript of the conversation appears below.

A Natural Experiment

The paper is about the summer youth employment program in New York City. We have data from 2005 to 2008. One of the nice things about the program is that applicants who apply get lotteried in to have access to a summer job. This is great for research purposes because we can compare basically identical people, one of whom happened to get access to a summer job and one of whom did not, and we can follow them from the year that they participated or don’t participate up through 2014. We match the data from the summer youth employment program with tax data, IRS administrative data and data from the New York State Department of Corrections and Community Supervision. We’re able to see the outcomes for earnings, college enrollment, incarceration and mortality for the youth who participate and who do not.

“We were particularly surprised about the mortality results: In essence, the program saved about 20 lives per year over the four years that we studied.”

The Goals of Jobs Programs

There are three rationales to have summer youth employment programs. One, you want to give youth a leg up and give them better access to jobs in the future. The second is that you want to transfer money directly to youth in the year of the program — so basically, put money into the hands of low-income kids. And the third is that you want to keep youth out of trouble, and putting them in a job might help that.

We analyzed the data to see if we could answer: Is the program accomplishing each of these three goals? The takeaways are it’s certainly transferring money to low income youth, because in the year of the program, the people who get access to the jobs earn significantly more than people who don’t. So it’s not as if the program is giving jobs to people who would have gotten jobs otherwise. The second rationale — that we’re putting youth on a better path so that they can earn more money in the future — we actually don’t find any evidence of that. In the three years following the program, the youth earned a little bit less if they had participated in the program. So we investigated that in a little greater detail because it was a little bit surprising to us.

Basically, we found that what matters is what job they get placed in for the summer of the program, because that job tends to be “sticky.” They take similar jobs like that afterward. The youth who end up being camp counselors or working in daycare centers, they tend to gravitate towards those kinds of jobs in the future, and those jobs on average earn a little bit less than other jobs. We find no evidence that the summer job lets them or helps them go to college at higher rates, we don’t see any effect on college enrollment. But by the fourth year after the program, they’re earning the same as the people who did not participate. So we don’t find great evidence for the program helping them earn more money later on.

But we found very strong effects on keeping the youth out of trouble. We actually found that the youth who get lotteried in to participate were more likely to be alive in 2014 and less likely to be incarcerated in the New York State prison system.

“What matters is what job they get placed in for the summer of the program, because that job tends to be ‘sticky.’”

Social Costs and Benefits

Among the most surprising things for us were the mortality and incarceration results. Those were things we were able to analyze because we had so many youth applying over the course of four years, and we could follow them for a long time. Other papers that studied similar programs hadn’t been able to look at so many people for so long.

We were particularly surprised about the mortality results: In essence, the program saved about 20 lives per year over the four years that we studied. That turns out to be quite important when you think about “Is the program working? What are the social costs of the program and the social benefits of the program? How do we compare those?” Because when you save lives, there’s lots of value, social value that you’re creating from the program.

Targeting for the Best Outcomes:

The great thing about this project is that it is the real world. These programs exist in New York City and in cities across the country, and so the results of the paper help give a sense of what the programs are actually achieving and maybe answer the question, should they be expanded? Which groups should they target? In the paper, we show the mortality, incarceration and earnings results for different subgroups. Maybe it’s worth targeting the programs or trying to reach out to youth for whom the benefits are greater.

Subtle Benefits

I think when people think about summer youth employment programs, they often think that their main contribution is helping people get jobs. One of the things we find that was so surprising to us is that, in fact, a lot of the benefits accrue on these other dimensions of keeping youth out of trouble, putting them on a safer path. They might have a different group of friends because they participated in the program, and those friends might keep them out of trouble and keep them alive. Thinking about all the ways in which the summer youth employment programs can help these youths is something that policy makers and others should keep in mind.

“The great thing about this project is that it is the real world. These programs exist and … the results of the paper help give a sense of what the programs are actually achieving.”

The Value of a Large Sample Size

One of the nice things about the data that we have to analyze is just how many people we’re able to look at. So we actually are following 200,000 unique individuals submitting 300,000 applications — some people apply multiple years. Having so much data allows us to talk about very rare outcomes like mortality — which, thankfully, is quite low in this population — but it allows us to make statistical claims about the lowering of the mortality risk for people participating in the program.

Next Steps

We’re following up on this paper with trying to better understand incarceration results, and merging in data on arrests and convictions in New York state to try to get a better sense of what’s driving that effect. There are also lots of other summer youth employment programs across the country, and one of the things that is on the horizon is helping other cities identify the strengths and weaknesses of their programs by doing similar evaluations on what is happening with the youth that get to participate in their city’s summer youth employment program.