AI, Data, and Predicting Urbanism: Interview with Peter Hirshberg and Anna Fedorova

Cities are among the most complex systems on earth. Over the past century administering their many functions –e.g. planning, permitting, logistics, operations–\xa0\xa0 has achieved tremendous urban development and flourishing, while also creating bureaucracies with their many rules, requirements, oversight mechanisms, and information silos. A key question today is: can these mechanisms scale or do they inevitably tend toward the slow and sclerotic?\xa0 This is all the more urgent in our post-pandemic era, given the various crises our cities face: housing is painfully unaffordable, downtowns’ land use needs reimagining as shopping and office workers’ activity withered, and homelessness continues to be a convoluted challenge. How do we make constructive decisions amidst so much complexity in an integrated way? The hope –and remarkable opportunity– here is that the convergence of emerging AI and data science technologies that take into account the multifaceted complexities of civic life might reduce bureaucracy, improve city governance, and serve as powerful new tools for urban planning and civic engagement.

Today’s high-performance computing powers include technologies like agent-based modeling, generative AI, complex system modeling, and AR/VR (we’ll discuss their capabilities below). When wielded wisely, these tools have unprecedented potential to strengthen our collective agency and accelerate improvements in urban life and governance. They can simulate real economic and behavioral data in civic contexts and quickly visualize intricate ideas, allowing us to demonstrate the practical impact of desired policies, and more powerfully inform and persuade communities to participate in civic decision-making. We might think of this as the “fifth estate” — a new form of collective agency enabled by more distributed (accessible) new technologies that help civil citizens and leaders alike to hold our institutions accountable and take collective action towards mutually beneficial outcomes.\xa0

What kinds of new tools are emerging?

Agent-based models are about to be the next technology revolution.\u202fIn economics, we have shown how agent-based models can make better real-time (before the fact) predictions than standard models.\u202fThis is just the tip of a large iceberg … -- Prof. J. Doyne Farmer, INET (Institute of New Economic Thinking), Oxford University

Historically, urban tools—from the observational tactics of William H. Whyte to primitive computer models, to the intelligent interfaces of Smart Cities—have helped us understand and shape urban environments. Yet throughout computing history, we haven’t been able to reliably predict specific policy outcomes in complex urban contexts. Today’s technologies are revolutionizing computational urban modeling and changing our understanding of what’s possible in urban governance and collaborative civic engagement. By unlocking powerful interactive and predictive capabilities, 21st-century digital tools are fueling the transition from classic collective intelligence to a new form—where civic problems are not merely observed and discussed but dynamically simulated and visually mapped to pave the way for better-informed communal actions and policymaking.

For example, Agent-Based Modeling enables digital simulations of real-world environments that are responsive to inputs from individual agents and dynamic interactions between them, including feedback loops, unintended consequences, system shocks, and limitations. By accurately representing behaviors and the interplay of individual agents and institutional entities (e.g. households, banks, businesses, etc) within complex socio-economic systems, ABM serves as a powerful predictive instrument in economics and policy research. This new approach colossally improves our capacity to provide reliable insights and understand the effects of complex policy interventions before implementing them in real life. A notable example: researchers at Oxford and the Bank of England have used agent-based modeling to study the effects of macroprudential policy interventions on the UK housing market.

Digital Twins are virtual replicas of physical entities, such as buildings, infrastructure, or even entire cities, used to simulate and analyze real-time processes and their performance in real environments. This allows urban planners and decision-makers to predict reactions to changes in physical environments and make decisions accordingly. Digital twins rely on IoT (Internet of Things) sensors and other data sources to continually update the virtual model, ensuring that it reflects the current state of its physical counterpart.

Additionally, generative AI and its visual capabilities (i.e. images, video), and AR/VR, enable novel ways to generate and visualize ideas. The ability to quickly illustrate concepts allows us to test and optimize them, plus communicate new possibilities, gather feedback, and more thoughtfully engage with local communities – all before any real-world application. Imagine doing this with rough yet elaborate drafts of public art concepts, buildings, rooftop gardens, urban landscapes and lifestyles, etc.\xa0\xa0\xa0\xa0

Can you go a bit deeper into how all this might work?

Let’s go through a few examples that demonstrate the transformative potential of these technologies.

1- Generative AI

We know generative AI because of popular large language models (LLMs) like Chat GPT and text-to-image tools like Midjourney. Government regulations – laws, building codes, and social benefit programs– are famously complex, often dense documents, a perfect use case for LLM’s. Over 500 governments use the Citizen Lab platform which incorporates AI to cluster, group, and organize public comments. At the Accelerate SF Hackathon architects asked if an AI might “read” the city’s building and fire codes and find where they contradicted each other; a prototype was developed over a weekend in November 2023 and presented to city supervisors. Tufts and Northeastern University professors developed a tool to allow families of special needs children to “have a conversation” with their individualized education program which otherwise exists as 50-100 page PDF with complicated legalese.

UrbanistAI is an AI visualization tool that uses the text-to-image capability to turn the public’s ideas for how public space might be used into realistic development renderings within the existing built environment. For example, this video illustrates ways in which public squares in Helsinki may be revitalized.

CityStructure is an AI-powered data science tool used to assess development opportunities for single-family and multi-family buildings in San Francisco and San Diego counties, revealing each parcel’s maximum height and value potential according to the latest regulations (e.g. possible ADUs, additions, potential to split the lot). CityStructure computationally tames wildly convoluted city ordinance data to determine homeowners’ maximum allowed building area, height (number of levels), number of units, and potential value added – including income-generating possibilities from available development options. Developed by Felicia Nitu at a hackathon and launched in early 2024, CityStructure‘s assessment saves thousands of dollars on architect fees on this crucial first step before engaging a contractor or architect in further planning and development process. Vastly simplifying and automating this step makes Development Analysis accessible to many more property owners, including those who may not have previously considered development opportunities. Thus, this technology can facilitate ( much ) faster production of sorely needed housing units.

The image below and this demo video illustrate the power of data science in transforming unwieldy city ordinance data to help homeowners assess and develop their properties to maximum potential in height and value, optimizing for their desired goals.

2- AR/R: Augmented and Virtual Reality technologies

Powerful visualizations in AR/VR make it easier for communities to get on the same page about proposed improvements in an urban environment. For example, New Rochelle’s city planners created AR/VR renderings of their downtown revitalization designs overlaid over existing maps of the area. These renderings allowed New Rochelle’s community to viscerally experience these possibilities vs lackluster existing reality, seeing the stark contrast between what is and what could be. More importantly, this approach gathered invaluable feedback from city residents, fueling equitable collaboration (= collective intelligence) and speeding up community consensus.

3- Convergence of high-performance computing technologies

At MIT’s City Science Lab, many of these techniques converge in project CityScope, an Urban simulation system directed by Kent Larson. Multiple urban stakeholders convene in front of a physical model of a neighborhood or development… made of Lego. Moving the Lego buildings or adding height to them changes factors in the models. Other legos are used as sliders to dial in transit or parking options, other than tradeoffs across residential, commercial, or retail usage. Displayed on a screen above the model are the implicit tradeoffs in traffic, housing density, diversity, day v nighttime uses, pollution levels etc. This combination of agent-based modeling, generative AI, and augmented reality tools has helped optimize urban planning in Hamburg, Germany, and Harvard and Kendall Squares in Cambridge, Massachusetts.

The most recent versions of the system synthesize entire populations. Starting with demographic profiles of a population and informed by mobility data (culled from carriers) and economic data (from MasterCard) the system can synthesize individual agents, simulate their behaviors and interactions, and over multiple generations (practically installed at machine speed)\xa0 these virtual agent populations behave similarly to their human counterparts. These can be used to predict the interrelated outcomes of a complex system:\xa0 public health, CO2 emissions per capita, different scenarios that solve for density, different logistics infrastructure alternatives, etc.

AI has taken over the conversation in the arts, tech, education, law/regulation, and even music. It hasn’t yet made a major mark in architecture, city planning, and urban governance, yet these disciplines are facing major crises that these tools can help resolve. How do we develop more housing, make sure it’s affordable, and how do we possibly agree on where to build it? How might we reduce bureaucracy? How do we reinvigorate downtowns after commercial and residential patterns have shifted?

Peter Hirshberg has led emerging media and technology companies at the center of disruptive change for more than 30 years. His sensibilities around how technology can empower people go back to his early days at Apple, where he headed up Enterprise Markets under Steve Jobs. He\'s the author of the best-seller "The Maker City: A Practical Guide to Reinventing Our Cities, which chronicles his work with the Obama White House fostering economic development in cities across America. Currently, he’s a principal of CoPlace, a 30-acre urban development in San Diego that is applying Maker City concepts to economic, workforce, and housing development in Southern California. In 2008 Hirshberg co-founded Gray Area, a pioneering creative hub for the integrated practice of art and technology, a San Franciscan institution globally recognized for its forward-looking programming around creative coding education and cultural incubation. His teams pioneered the use of civic open-data projects in San Francisco; he believes today\'s Al/lIm technologies will similarly transform civic IT and data assets making them far more useful to citizens, planners, and policymakers. Hirshberg serves on the board of San Francisco\'s international economic development agency GobalSF, and is a Senior Fellow at the USC Annenberg Center on Communication Leadership and Policy and a Henry Crown Fellow at the Aspen Institute.

Anna Fedorova is a seasoned product management professional with a strong track record in driving innovation, growth, and strategic solutions at tech startups that transform industries. In this role over the past decade, Anna helped build digital platforms in telehealth, e-commerce and two-sided marketplaces, productivity and predictive analytics tools, construction AI, and social and gov tech. Her work spans developing technologies from concept to launch, which includes research, analytics, strategy, and combining user-centric design principles with agile methodology practices. She also advises or consults on all of the above.\xa0 Anna’s earlier background involves an Economics degree at UC Berkeley, and research and analyst work at an economic consultancy in high-stakes litigation and finance. Currently, she is exploring impactful solutions to the California housing crisis. Anna is passionate about understanding and addressing complex civic issues by applying her practical experience in designing efficient systems and processes, along with big-picture thinking and knowledge of policy and market dynamics. Anna’s keen interest in political economy, architecture, and urbanism drives her motivation to improve the civic experience in places she calls home — namely, the San Francisco Bay Area and Los Angeles. She aspires to apply her analytical and entrepreneurial savvy to real estate development and policy, knowing that expanding access to affordable housing will create more vibrant, safe, efficient, and prosperous cities that provide the synchronistic spontaneity that makes urban environments beautifully special.

"AI, Data, and Predicting Urbanism: Interview with Peter Hirshberg and Anna Fedorova" is written by architect John Marx, AIA, the founding design principal and Chief Artistic Officer of Form4 Architecture, an award-winning San Francisco-based firm that designs prominent buildings, campuses, and interiors for Bay Area tech companies such as Google and Facebook, laboratories for life-science clients, and workplaces for numerous other companies. In 2000-2007, Marx taught a course on the topic of placemaking in cyberspace at the University of California, Berkley, and in 2020 he designed his first project in the Metaverse for Burning Man: The Museum of No Spectators. The following year, John Marx led a design team charged with creating a $500B portal to the Metaverse.