Many things came together for nationalism to succeed as the world's organizational system.
One big change – print-capitalism emerged. Languages were formalized. Before, all materials were hand-written and costly, and most people could not write - so most of the world didn't have written documents. With the printing press came social and institutional changes, including the creation of standardized languages – "languages of states" used for bureaucratic purposes. The languages themselves, and the printed materials that were disseminated throughout 'states,' created what Benedict Anderson calls 'imagined communities.' Now you could envision yourself belonging to a political community larger than just the people you saw in your own town. People started to develop a sense of belonging to people they had never met – in the idea that they all belonged to the same 'state.' This was a powerful change. It lent a sense of antiquity to these communities – as if the members of these states had always belonged to each other in this way – as if they hadn't just been recently linked through a bureaucratic system. This mattered. Think loyalty. Think upcoming wars . . .
The first wave of nationalism came from European colonists in the Spanish-American colonies. After the colonists gained control over land in the Americas (by killing and enslaving the indigenous peoples) they desired independence from their European leaders. Print-capitalism, newspapers specifically, played a major role in constructing a sense of community and nationhood among the colonists. Their movements to break away from their European leaders were national in character. Independent 'states' formed in the Americas. Then came the French revolution at the end of the 18th century – creating the first European state.
These states were strictly political/territorial. They were not tied to a 'nation' in the sense of nation-as-a-people. (With 'people' meaning a group with shared history, language, and culture). The state was a political entity, and its focus was its national economy. While states invariably involved democratization and electoralization, they were also about expansion – swallowing up larger and larger masses of land and people. As the states expanded in size, they encompassed different 'peoples' or 'nations' – groups distinguished along ethnic and/or linguistic lines. At this time it was generally accepted that states would be heterogeneous – made up of many peoples. The boundaries of 'states' and 'nations' did not line up in any clear way.
In this age of secularization, democratization, and electoralization, monarchies were losing political legitimacy, and control, fast. They had to jump on the bandwagon and declare their monarchies were 'states.' In a far cry from the popular movements that had come before, monarchs created official nationalisms – designed to engender loyalty from the masses.
This official nationalism tied 'state' to 'nation-as-people' for the first time. Languages-of-state, used for bureaucratic purposes, took on tremendous power as state bureaucracies expanded like crazy. Newly-developed national school systems tied language to access to power. And in an age of growing unrest and widespread mobilization of poor and working class people, this new nationalism provided a social identity for all of those people between working class and ruling class. The new "ethno-linguistic nationalism" created a link between the middle classes and the upper classes. This was the first development towards modern race-thinking in Europe. It was also the start of the tenuous relationship between socialism and nationalism – sometimes conflicting, sometimes reinforcing.
These new states became empires – establishing colonies all over the globe. This had a major effect on the psyches of Europeans. The growing power of the imperial states and the growing division between the colonized people and the colonizers all reinforced the sense of "us vs. them" and thus reinforced the conflation of nation-hood with people-hood. This pleased the rulers just fine.
This all leads up to WWI. In the meeting that ended the war, the "nation-state" became the only legitimate political entity. United States President Wilson decreed in his Wilsonian Principle that state boundaries should coincide with nationality and language. The multi-national empires that were heterogeneous by definition (like the Ottoman Empire – one of the war's losers) were divided up – supposedly along lines of nationality and language. But since all nations were heterogeneous to one degree or another – the logical implication of this policy was mass expulsions and exterminations to create "pure" nation-states. (See the next 90 years.)
This was the end of the "expansion" period of nationalism. From this point on, existing nations either stayed the same size, or divided into smaller nations. Imperialism didn't go away, it just changed forms.
This new concept of nation-state was used by colonized, oppressed, and/or unrecognized peoples to advocate for their right to statehood. Anti-imperial movements gained traction in colonized territories. Back in Europe, official nationalisms took hold with a vengeance. In states defeated in WWI, militant nationalism developed into fascism, and in came WWII.
After WWII, the anti-imperial movements throughout the globe gained power. State creation at this time was about decolonization, revolution, and unfortunately, intervention of outside powers. Colonized people revolted against the colonizers, but the new nations they developed had many of the short-comings of the American and European states they were modeled after, and suffered the social and economic consequences of having been colonies.