On the basis of a randomized controlled trial, we evaluate a scholarship program in Mexico, Programa de Becas Educación Media Superior (PROBEMS), aimed at improving graduation rates and test scores among upper secondary school students from poor backgrounds. We find that, on average, the program has no effect on either graduation rates or math and Spanish test scores. We point to two possible reasons for this failure: (i) the program was badly targeted, with many of the recipients being from less disadvantaged families than intended; and (ii) the prior academic achievement of those eligible was often insufficient for successful completion of the academic requirements of upper secondary school. This points to accumulated achievement deficits that could be addressed by interventions targeting learning at an earlier stage.