Most of the world’s most successful urban areas today share a common ailment: a housing shortage.

Put simply, wealthy cities like London, New York City, San Francisco and Paris don’t have as much housing as they need to support their growing populations. Rents and home prices have surged as a result, pushing the poorest out of their homes, forcing the working class into smaller and more crowded accommodations and sapping wage gains for upper-middle-class professionals.

homes in central London
A row of homes in central London. (Photo by Oli Scarff/Getty Images)

The housing shortage is largely a product of the post-industrial economy and its polarised fortunes. High-income jobs have clustered in a handful of metropolitan areas, concentrating people and wealth while exacerbating economic and geographic inequalities. Meanwhile, rising construction costs and restrictive regulations have made it increasingly difficult to build new homes in these often already densely populated areas.

In the US, New York, San Francisco, Seattle, Boston and Washington, DC, are among the economy’s major winners, with housing prices to match. Canadian cities like Vancouver and Toronto have seen similar dynamics develop, and the UK’s fortunes are infamously concentrated around London. Even Germany, which has retained a more geographically diffuse manufacturing economy, has seen an increasing concentration of economic activity and rising housing costs in cities like Berlin.

While the housing shortage is a defining characteristic of modern wealthy cities, it doesn’t have to be this way. There are growing movements calling for new policies and fast action to address rising housing costs, and experts point to a few notable examples where they say wealthy nations have managed to avoid a housing shortage altogether.

But while political leaders increasingly acknowledge the extent of the housing shortage, policy efforts to address the issue have faced crippling opposition. Covid-19, meanwhile, could make everything worse. Many housing experts expect construction to slow as a result of the pandemic, thus further constraining the supply of homes.

The origins of the modern housing shortage

The last time the world’s wealthy nations faced widespread housing shortages was after the Second World War. In countries such as the Soviet Union, Germany and Japan, the conflict leveled cities and destroyed great swaths of housing stock. Even in relatively unscathed nations like the United States, non-military construction languished during the war. In 1944, only 114,000 new homes were started in America.

After the conflict, governments of all political persuasions stimulated a massive construction effort: by 1950, there were 1.7 million new homes started in the United States and about 250,000 in the United Kingdom. This building boom lasted, with some hitches, until the 1970s. (Housing starts measure when construction of a new home  begins. Projects are often delayed and can take years to work through permitting, so it makes sense to count a project as meaningfully contributing to the economy when foundations are laid and work actually begins.)

The industrial economy was geographically dispersed within nations, which helped ensure that opportunity – and demand for housing – was not bottled up in a handful of cities or regions.

That didn’t mean the opportunity was equally shared. In the United States, a web of systemically racist lending practices, exclusionary zoning codes and boundaries policed by force kept nonwhite Americans – and especially Black residents – segregated from the economy’s fruits. It was exceedingly hard for Black Americans to access new suburbs, many of which explicitly barred access to that demographic people (and sometimes Jews). Most European and East Asian societies did not have large racial minorities, although as they became more diverse in the late 20th century, many non-majority groups became segregated in countries like France, as well.

As the manufacturing economy dwindled and remaining factories moved to more-spacious locales on the suburban fringe, cities became less polluted and more appealing to the middle class. (In the United States, the precipitous decline in crime after the early 1990s abetted this trend.) Meanwhile, as higher-income people delayed getting married and starting families, the number of households increased even as urban populations stagnated or declined.

All of this contributed to increased competition for scarce housing in cities where the best-paid work could be had in abundance. Prices soared, gouging the wages of even highly paid white-collar tech and finance workers. But the problem was far more acute for low-income and working-class residents. Their wages had been crunched by the decline of the labour movement, but they still needed to live close to the urban core and the plentiful – if low-paid – service sector work there.

“The thing that’s expensive isn’t just a roof over your head, it’s having that in a place that people want to live in,” says Rick Jacobus, a strategic adviser with the Grounded Solutions Network. “It’s the places with the high-paying jobs that are experiencing this problem most acutely. It’s a supply-of-good-places problem, not just a supply-of-houses problem.”

In the US, “urban renewal” programs of the mid-20th century demolished many older urban neighbourhoods, gutting the supply of more-affordable homes along the way, resulting in far greater dangers of eviction and homelessness. Beginning in the 1980s, the number of Americans without a reliable place to live began increasing. Although President Ronald Reagan argued that the rise of homelessness could be attributed to individual causes like mental health and substance abuse, the academic literature shows that the fault lay mostly with structural issues like vanishing affordable housing and stagnating wages.

downtown San Francisco
Downtown San Francisco from above. (Photo by Josh Edelson/AFP via Getty Images)

As Conor Dougherty documents in his book Golden Gates, by the late 1970s, the San Francisco Bay Area was already developing a housing shortage. Future Harvard professor Lawrence Katz explained in his 1981 commencement address at the University of California, Berkeley, that sky-high prices couldn’t be attributed solely to high demand. After all, in the 1950s and 1960s, demand had been even higher and prices had remained affordable because new housing continued being added to the market at a healthy clip.

Nimbys vs. Yimbys: Can upzoning solve the housing shortage?

Today, debates about building more housing are commonly framed as a battle between Nimbys and Yimbys. Nimby stands for “not in my backyard,” and it’s generally a pejorative label applied to homeowners who oppose new construction. As the housing crisis has intensified in recent years, a countermovement of Yimbys (“yes in my backard”) has emerged to advocate not just for new housing but also for policies that make it easier to build – and harder to block – new housing.

A central policy goal of the Yimby movement is “upzoning”, or changing zoning codes to allow for a greater density of homes. That can be achieved by allowing, for instance, taller buildings, or more units per lot, or by undoing policies that prohibit anything other than single-family homes. By making it easier to build apartments, triplexes and other forms of multifamily housing, they hope to create a more diverse housing stock that will be affordable to more people.

Academic research broadly affirms their argument. A 2019 paper by Xiaodi Li studied high-rise residential construction in New York City and found that for every 10% increase in housing stock, rents decreased by 1% and sale prices fell within a 500ft radius. Another 2019 paper, from the Upjohn Institute for Employment Research, found that new market-rate buildings decrease nearby rents from 5% to 7%. In a separate Upjohn study, Evan Mast found that new construction mostly attracts residents from elsewhere in the city, often pulling higher-income residents from downmarket areas where they then free up housing that’s more affordable.

But ensuring additional supply is not a simple feat. Upzoning does not necessarily mean that more units will be built. In 2018, Minneapolis enacted a law that eliminated single-family zoning and allowed triplexes in any area formerly zoned in that fashion. But by September 2020, only three such units had been created, as other kinds of building restrictions and financing limitations have prevented developers from pursuing such projects.

The Urban Institute’s Yonah Freemark found, in a 2019 study, that Chicago neighbourhoods that were upzoned did not see an increase in construction or density in the short term. Instead, land values increased and speculators swooped in, creating negative effects for neighbourhood residents. A 2020 report from Andrés Rodríguez-Pose and Michael Storper argues that “blanket upzoning” – without protections in place for lower-income residents or affordability requirements – “are likely to reinforce the effects of income inequality rather than tempering them”.

Critics of upzoning similarly say adding more supply doesn’t meaningfully ease price pressures. Neighbourhood activists and historic preservationists often argue that adding new homes will disrupt existing communities, change their character and undermine what residents find valuable there. Some renters and anti-gentrification advocates additionally fear that new construction brings pricier homes, exacerbating price pressures and affordability issues by attracting more high-income residents.

The case for housing subsidies and public housing

There is broad agreement that public subsidies are required to address the affordability challenges facing the lowest-income people. All but the most committed libertarians acknowledge that more homes in high-demand areas won’t do much for those living in poverty. In the United States, with its threadbare safety net, roughly three quarters of those who are eligible to receive assistance with their housing costs do not get help from the government.

“[N]ew market-rate housing is necessary but not sufficient,” the Furman Center concluded in an analysis of studies about the effect of more housing supply on prices. “Government intervention is critical to ensure that supply is added at prices affordable to a range of incomes.”

apartment buildings in New York
New York City had one of America’s most successful social housing systems until Congress systemically deprived it of funding. (Photo by Spencer Platt/Getty Images)

Many wealthy democracies have cut back on subsidised housing in recent decades. Conservative governments in the US and the UK undercut their postwar public housing programs in the late 20th century, eliminating the supply of new government-owned units. Instead, they’ve turned to rental assistance programmes and public-private partnerships to address affordable housing. Those efforts have been critiqued for not providing enough support to the poorest and for directing some money to private actors.

The perils of too much local control

In many English-language nations, local officials have a great level of authority over planning and development decisions. In the United Kingdom, for example, while local officials have relatively little authority overall, they have immense power over real-estate development. Landowners and developers pitch projects on a case-by-case basis, where both planners and politicians are given veto power.

“There’s no rules or map about what you can build on a site. Instead, it’s this protracted negotiations process,” says Anthony Breach, a housing and planning analyst with the Centre for Cities in the UK. “It’s really not a very good way to run any part of the economy, but for housing it’s particularly bad.”

American and Canadian cities have zoning codes that give power to local officials, but in many localities there are ways to build without this kind of exacting oversight. In some cases, however, when constituents and their elected officials do not want neighbourhood change or new construction, they can make their rules too strict to allow new development (a practice known as exclusionary zoning).

On the other hand, some wealthy nations have handled the housing challenge better than others, with alternative planning institutions and a healthy supply of new homes that have kept housing costs from raging out of control.

In Singapore, the authoritarian government controls much of the housing production on the limited available land. A central development agency builds new homes, usually in soaring towers, and then sells them to residents at an affordable rate and with a 99-year lease. The government then retains control of the land itself, and at the end of the lease, the building goes back to public ownership. It can then be redeveloped as a larger building to ensure that housing capacity grows over time.

A more germane example may be Japan, which has institutional and political norms more comparable to other wealthy democracies. Its zoning laws are set at the national level and then applied by local governments. That eliminates the patchwork of confusing codes that help make building in many English-speaking nations so challenging.

Japanese zoning laws are also much less restrictive, allowing residential and neighbourhood-scale commercial construction almost anywhere except in industrial areas. Greater densities are allowed as well, while historic preservation is a much less cherished value. All of this has combined to make Tokyo far more accessible to the working and middle classes than London or New York.

Tokyo view of Mount Fuji
Tokyo with Mount Fuji in the background. (Photo by Kazuhiro Nogi/AFP via Getty Images)

“Housing in Tokyo, even though it’s a fast-growing city with tons of opportunity, is much, much more affordable than housing in its peer cities around the world,” says Emily Hamilton, director of the Urbanity Project at the Mercatus Center at George Mason University. “Households across Japan spend less of their income on housing, compared to households across the US.”

How could other wealthy nations become more like Japan? The most important element, Hamilton argues, would be to lessen the discretionary power of local authorities over zoning, lot sizes and other building regulations. There have been gestures in this direction. In the UK, Boris Johnson’s government has put forward a proposal for accelerating development with “automatic” permissions in high-growth areas. In the United States, Democratic Party politicians have proposed withholding federal transportation funding from municipalities that don’t change their exclusionary building regulations.

Without that kind of federal intervention, it is unclear if US states could overcome obstacles to local development on their own. California failed to adopt a raft of housing reform legislation in 2020, but the state is distinguished by the fact that it is at least debating the issue. In many other high-cost jurisdictions, like New York and the Washington metropolitan area, there have been few serious efforts to ease development restrictions and address the housing shortage in the most opportunity-rich corners of America.