Chicago, like most large cities in the United States, has been struggling to reduce youth gun violence for decades. Cities all across the country have responded by implementing a variety of new programs and strategies over the years, all with the hope of identifying effective and lasting approaches to reduce the number of young people involved with gun violence as either victims or offenders and to improve the life outcomes for those at most risk. Nevertheless, youth gun violence remains a chronic problem across the country, in part because most new programs to combat gun violence are implemented in ways that make it difficult to conduct rigorous evaluations and hence to identify the effectiveness of these efforts. This limits the ability of government agencies and their community partners to either learn from experience or share lessons learned about best practices across jurisdictions.
The University of Chicago Crime Lab launched a design competition to identify promising intervention ideas and then to work with one or more selected applicants to raise private funding in order to launch and rigorously evaluate one or more new pilot programs.
The aim of the design competition is to produce a program with the potential to generate evidence about what works, for whom, and why, that is as reliable as what is gathered in clinical trials in medicine—another area where lives are at stake. That evidence in turn can help inform large-scale violence prevention efforts in Chicago and across the country. We initially sought letters of interest from organizations and agencies that have promising ideas for programs that aim to reduce youth involvement with gun violence. We were particularly interested in programs that seek to achieve this end by strengthening incentives and opportunities for the highest-risk youth to engage in such prosocial, positive activities as schooling, as well as programs that seek to reduce youth involvement with firearms.
Evaluations will be carefully structured to not only determine the overall effectiveness of the program selected, but also to determine who benefits most from the program and why, with an eye towards improving program designs and enhancing impacts. For this reason the Crime Lab required applicants to indicate how they would implement their program in a way that can be rigorously evaluated.
The Importance of Rigorous Evaluation
A total of 510 people were murdered in Chicago during 2008. Eighty percent of these victims were killed by gunfire, the vast majority were male, and nearly half were between the ages of 10 and 25. The dramatic overrepresentation of both young males and firearms in homicide is not unique to Chicago, nor are these patterns new. Yet over the past 50 years, our society has made far less progress in understanding how to protect people from gun violence (and violence more broadly) than from other serious threats to life and health. From 1950 to 2005, the overall age-adjusted death rate in the United States declined by nearly 45 percent, from 1,446 to 799 deaths per 100,000 people. This decline was driven in large part by massive drops in deaths from such major illnesses as heart disease and cerebrovascular diseases (stroke). In contrast, despite some cyclical variations, the murder rate in 2005 remained about 20 percent higher than its 1950 value.
Why have we made such dramatic progress in reducing deaths from disease while homicide remains such a persistent problem throughout the United States? We believe one answer is that data and evidence are generally taken more seriously in medicine than in the area of violence. Before new medical innovations are used by the public, the United States Federal Drug Administration typically requires a rigorous series of randomized trials to determine whether these interventions are actually effective. In contrast, federal, state, and local governments throughout the United States have implemented a wide variety of innovative programs to reduce youth gun violence over the past 50 years—but almost never in a way that can be rigorously evaluated. The logic behind these programs often seems quite promising. Claims of dramatic success are not in short supply. Yet the youth gun violence problem remains. Progress in addressing youth gun violence in Chicago, or anywhere, is extremely difficult without guidance about what programs work, for whom, why, and how they can be improved.
There is strong consensus within the research community, supported by the National Academy of Sciences and the Council for Excellence in Government, that scientifically rigorous methods—including the use of fair, random lotteries to allocate people or places to receive program services in cases where there is “excess demand” for help—should be used to gather credible information about whether a program strategy works or not.3 This approach allows researchers to quantify the value of a program—for example, whether it increased high school graduation rates or reduced gunshot injuries by 3 percent versus 20 percent—which in turn enables policy makers to allocate resources to those programs that generate the largest benefits to our children and our city overall for a given level of expenditure. Rigorous evaluations are also characterized by careful attention to the identification of the population subgroups for whom the program works best, and they focus on understanding program implementation, which is key to future replication.
The programmatic focus of the original design competition was driven by the findings of the Crime Lab’s report, Gun Violence among School-Age Youth in Chicago, which may provide useful background to prospective respondents about key risk factors for youth gun violence offending and victimization, as well as promising intervention strategies. This study was conducted by the Crime Lab to learn more about the context of Chicago youth gun violence and to listen to community perspectives on the nature of the problem and initial ideas about promising or desired intervention strategies. The report’s findings are based on a series of extensive focus groups with youth, parents, social service providers, community leaders, and other stakeholders, as well as on analysis of a variety of government administrative and other data sources. Copies of the report are available here.
To be considered for the design competition, all applicants were be required to:
- Focus on school-age youth ages 10 to 21 who have previously been involved with the criminal justice system in some way and who live in communities with high levels of violent crime.
- Focus on increasing engagement in school and school performance for those youth at highest risk of involvement with gun violence and/or on limiting youth involvement with firearms.
- Demonstrate a capacity for providing service to at least 400 individuals (or at least 40 schools or neighborhoods, for programs that are focused on providing services to larger social or geographic entities beyond the individual youth).
- Present a clear description of the problem the intervention seeks to address and include any relevant data to support the magnitude and scope of the problem and likely demand for the intervention.
- Discuss how the intervention program either might improve the life chances of young people or otherwise change the incentives associated with socially desirable or detrimental behavior, or how it might reduce gun misuse. Please include any supporting research or evidence.
- Designate a "lead" applicant to submit the letter of interest (in cases where multiple organizations and/or agencies are collaborating) and include a list of agencies and organizations that would partner in the intervention (if applicable). Incorporate an estimated project timeline.
- Include an estimate of the overall annual project cost to conduct the intervention. This should not include the cost of the evaluation.
- Have been in existence for at least two years.
- Be willing to work with the Crime Lab and structure the pilot intervention so that it can be rigorously evaluated.
- Produce audited financial reports for the last two years (only if a full proposal is invited).
Given the scale of the youth gun violence problem, no new pilot program could hope to serve all youth, families, schools, or neighborhoods in Chicago that are eligible and in need. Respondents to this request for letters of interest were urged to consider the use of fair, random lotteries to allocate services in cases where there is excess demand for program slots. Fair, random lottery procedures have been adopted by housing agencies across the country to allocate scarce housing vouchers or other program benefits to much larger pools of eligible program applicants, as well as by school systems across the country to allocate enrollment slots in oversubscribed charter schools or as part of other public school choice plans. The use of random lotteries to allocate services when all eligible program applicants cannot be served has the virtue of being fair and helps to facilitate the most rigorous possible evaluation to generate strong scientific evidence about the program’s effectiveness.