Cities generate negative, as well as positive, externalities; addressing those externalities requires both infrastructure and institutions. Providing clean water and removing refuse requires water and sewer pipes, but the urban poor are often unwilling to pay for the costs of that piping. Standard welfare economics teaches us that either subsidies or Pigouvian fines can solve that problem, but both solution are problematic when institutions are weak. Subsidies lead to waste and corruption; fines lead to extortion of the innocent. Zambia has attempted to solve its problem with subsidies alone, but the subsidies have been too small to solve the “last-mile problem” and so most poor households remain unconnected to the water and sewer system. In nineteenth-century New York, subsidies also proved insufficient and were largely replaced by a penalty-based system. We present a model that illustrates the complementarity between infrastructure and institutions and provides conditions for whether fines, subsidies or a combination of both are the optimal response. One point of the model is that the optimal fine is often not a draconian penalty, but a mild charge that is small enough to avoid extortion.