Starting around 3,000 years ago, a wave of innovation began to sweep through human societies around the globe. For the next millennium the continued emergence of new technologies had a dramatic effect on the course of human history.

This era saw the advancement of the ability to control horses with bit and bridle, the spread of iron-working techniques through Eurasia that led to hardier and cheaper weapons and armor and new ways of killing from a distance, such as with crossbows and catapults. On the whole, warfare became much more deadly.

During this era, many societies were consumed by the crucible of war. A few, though – the Achaemenid Persian Empire, the Roman Empire and Han China – not only survived, but thrived, becoming megaempires encompassing tens of millions of people and controlling territories of millions of square miles.

So what drove this cascade of technological innovation that literally changed the course of history?

We are a complexity scientist, Peter Turchin, and a historian, Dan Hoyer, who have been working since 2011 with a multidisciplinary team to build and analyze a large database of past societies. In a new paper published in PLOS One on Oct. 20, 2021, we describe the main societal drivers of ancient military innovation and how these new technologies changed empires.

A database for human history

The store of knowledge about the past is truly enormous. The trick is to translate that knowledge into data that can be analyzed. This is where Seshat comes in.

The Seshat Databank is named after Seshat, an ancient Egyptian goddess of wisdom, knowledge and writing. Founded in 2011 as a collaboration among the Evolution Institute, the Complexity Science Hub Vienna, the University of Oxford and many others, Seshat aimed to first systematically gather as much knowledge about humanity’s shared past as possible. Then our team formatted that information in a way that allows researchers to use big-data analytics to look for recurrent patterns in history and test the many theories aiming to explain such patterns.

The first step in this process was to develop a conceptual scheme for coding historical information ranging from military technology to the size and shape of states to the nature of ritual and religion. The database includes over 400 societies across all world regions and ranges in time from roughly 10,000 B.C. to A.D. 1800.

In order to trace the evolution of military technologies, we first broke them down into six key dimensions: hand-held weapons, projectiles, armor, fortifications, transport animals and metallurgical advances. Each of these dimensions was then further divided into more specific categories. Altogether we identified 46 such variables among the six technological dimensions.

For example, we distinguish types of projectile weapons into slings, simple bows, compound bows, crossbows and so on. We then coded whether or not each historical society in the Seshat sample wielded these technologies. For example, the earliest appearance of crossbows in our database is around 400 B.C. in China.

Of course, humanity’s knowledge of the past is imprecise. Historians may not know the exact year crossbows first appeared in a particular region. But imprecision in a few cases is not a serious problem given the staggering amount of information in the database and when the goal is to discover macrolevel patterns across thousands of years of history.

Competition and exchange drive innovation

In our new paper, we wanted to find out what drove the invention and adoption of increasingly advanced military technologies around the globe during the era of ancient megaempires.

Utilizing the massive amount of historical information collected by the Seshat team, we ran a suite of statistical analyses to trace how, where and when these technologies evolved and what factors seemed to have had the largest influence in these processes.

We found that the major drivers of technological innovation did not have to do with attributes of states themselves, like population size or the sophistication of a governance. Rather, the biggest drivers of innovation appear to be the overall world population at any given time, increasing connectivity among large states – along with the competition that such connections brought – and a few fundamental technological advances that set off a cascade of subsequent innovations.

Let’s illustrate these dynamics with a specific example. Around 1000 B.C., nomadic herders in the steppes north of the Black Sea invented the bit and bridle to better control horses when riding them. They combined this technology with a powerful recurved bow and iron arrowheads to deadly effect. Horse archers became the weapon of mass destruction of the ancient world. Shortly after 1000 B.C., thousands of metal bits suddenly appeared and spread within the Eurasian steppes.

Competition and connection then grew between the nomadic people and the larger settled states. Because it was hard for farming societies to resist these mounted warriors, they were forced to develop new armor and weapons like the crossbow. These states also had to build large infantry armies and mobilize more of their populations toward such collective efforts as maintaining defenses and producing and distributing enough goods to keep everyone fed. This spurred the development of increasingly complex administrative systems to manage all these moving parts. Ideological innovations – such as the major world religions of today – were also developed as they helped to unite larger and more disparate populations toward common purpose.

Within this cascade of innovation we see the origins of the world’s first megaempires as well as the rise and spread of world religions practiced by billions of people today. In a way, these critical developments can all be traced back to the development of the bit and bridle, which allowed riders better control of horses. Each step in this line has been long understood, but by employing the full range of cross-cultural information stored in the Seshat databank, our team was able to trace the dynamic sequence tying all these different developments together.

Of course, this account gives a greatly simplified explanation of very complex historical dynamics. But our research exposes the key role played by the intersocietal competition and exchange in the evolution both of technology and of complex societies. Although the focus of this research was on the ancient and medieval periods, the gunpowder-triggered military revolution had analogous effects in the modern era.

Perhaps most importantly, our research shows that history is not “just one damn thing after another” – there are indeed discernible causal patterns and empirical regularities through the course of history. And with Seshat, researchers can use the knowledge amassed by historians to separate theories that are supported by data from those that are not.

This story was co-authored by Daniel Hoyer, a researcher and project manager at the Evolution Institute and a part-time professor at George Brown College.

[Understand new developments in science, health and technology, each week. Subscribe to The Conversation’s science newsletter.]