Innovation is critical to economic growth, progress, and the fate of the planet. Although innovation may seem to happen at random, planners and politicians could take advantage of patterns that emerge to encourage innovation and growth. 

One emergent pattern, spanning centuries, is that the pace of innovation is perpetually accelerating, driven by a combinatorial explosion of possible recombinations of good ideas that accumulate over history. And that is why cities are the crucible of innovation. 

Geoffrey West of the Santa Fe Institute argues that cities are an autocatalytic attractor and an amplifier of innovation. On average, people are more innovative and productive when they live in a city because ideas can cross-pollinate more easily. Matt Ridley calls it “ideas having sex”. This positive network effect drives a positive feedback loop – attracting the best and the brightest to flock to the salon of the mind, the memeplex of modernity. 

Why does this drive innovation and accelerating change? In The Nature of Technology, Brian Arthur argues that ’all technologies are combinations of technologies that already exist.’ In any academic field, today’s advances are built on a large edifice of history. This is the foundation of progress, something that was not so evident to the casual observer before the age of science. Science tuned the process parameters for innovation, and became the best method for a culture to learn. 

The number of possible idea groupings grows exponentially as new ideas come into the mix (Reed’s Law). This explains the innovative power of urbanisation and networked globalisation. And it explains why interdisciplinary ideas are so powerfully disruptive; islands of cognitive isolation such as academic disciplines are vulnerable to disruptive memes, much like South America was to smallpox from Cortés and the Conquistadors. 

So what evidence do we have of accelerating technological change? At Draper Fisher Jurvetson, we see it in the diversity and quality of the entrepreneurial ideas arriving each year across our global offices. Scientists do not slow their thinking during recessions. 

For a good mental model of the pace of innovation, consider Moore’s Law in the abstract – the annual doubling of computer power or data storage. As Ray Kurzweil has plotted, exponential progress spans from 1890 to 2012, across countless innovations, technology substrates, and human dramas – with most contributors completely unaware that they were fitting a pattern. 

Moore’s Law is a primary driver of disruptive innovation – such as the iPod usurping the Sony Walkman franchise. And now it drives not only IT and communications, but also genomics, medical imaging and the life sciences. As Moore’s Law crosses critical thresholds, what was formerly a lab science of trial and error experimentation becomes a simulation science – accelerating the pace of progress and creating opportunities for new entrants in new industries. As a result, the industries effected by the latest wave of tech’ entrepreneurs are more diverse, and an order of magnitude larger – including everything from automobiles and rockets to energy and chemicals. 

Biology is in the midst of this transformation; we are actively reengineering the information systems of biology and creating synthetic microbes whose DNA was manufactured from bare computer code and an organic chemistry printer. But what should we build? So far, we largely copy large tracts of code from nature. But the question spans across all the complex systems that we might wish to build, from cities to designer microbes, to computer intelligence. 

As these systems transcend human comprehension, will we continue to design them or will we evolve them? As we design for evolvability, the locus of learning shifts from the artifacts to the process that created them. There is no mathematical shortcut for the decomposition of a neural network or a genetic program, no way to ‘reverse evolve’ with the ease that we can reverse engineer the artifacts of purposeful design. (My Google Tech Talk goes into some detail on the dichotomy of design and evolution.)

And what about human social systems? The corporation is a complex system that seeks to perpetually innovate. Leadership in these complex organisations shifts from direction setting to a wisdom of crowds, where the locus of learning shifts from products to process.  The lessons learned so far are a bit counterintuitive to some alpha leaders: cognitive diversity is more important than ability; disagreement is more important than consensus; voting policies and team size are more important than the coherence or comprehensibility of the decisions; and tuning the parameters of communication (frequency and fanout) is more important than charisma.

The same could be said for urban planning. How will cities be built and iterated upon? Who will make those decisions and how? We are just starting to see the shimmering refractions of the hive mind of human culture, and now we want to redesign the hives themselves to optimise the emergent complexity within. Perhaps the best we can do is set up the grand co-evolutionary dance, and listen carefully for the sociobiology of supra-human sentience.