Katja Grace at Meteuphoric:

In order to grow more powerful than everyone else you need to get significantly ahead at some point. You can imagine this could happen either by having one big jump in progress or by having slightly more growth over a long period of time. Having slightly more growth over a long period is staggeringly unlikely to happen by chance, so it needs to share some cause too. Anything that will give you higher growth for long enough to take over the world is a pretty neat innovation, and for you to take over the world everyone else has to not have anything close. So again, this is a big jump in progress. So for AI to help a small group take over the world, it needs to be a big jump.

Notice that no jumps have been big enough before in human invention. Some species, such as humans, have mostly taken over the worlds of other species. The seeming reason for this is that there was virtually no sharing of the relevant information between species. In human society there is a lot of information sharing. This makes it hard for anyone to get far ahead of everyone else. While you can see there are barriers to insights passing between groups, such as incompatible approaches to a kind of technology by different people working on it, these have not so far caused anything like a gap allowing permanent separation of one group. ...

Read the rest here

Some thoughts: a lot of these issues have been hashed out on the internet before. Making reliable predictions about the future is hard, and high quality debate about futuristic scenarios seems hard to do. High-quality criticism of singularitarian ideas is also hard to come by, so this post seems encouraging.

Moving to the object-level, a criticism. Consider:

Some species, such as humans, have mostly taken over the worlds of other species. The seeming reason for this is that there was virtually no sharing of the relevant information between species. In human society there is a lot of information sharing. This makes it hard for anyone to get far ahead of everyone else. While you can see there are barriers to insights passing between groups, such as incompatible approaches to a kind of technology by different people working on it, these have not so far caused anything like a gap allowing permanent separation of one group.

and translate it one step backwards in the history of the world:

Some stable patterns, such as life, have somewhat taken over the world of other stable patterns, at least on the surface of earth. The seeming reason for this is that there was virtually no correlation of relevant information (about which patterns are likely to stick around in the current environment) between life and nonlife. Life makes incremental improvements, nonlife executes some random walk or just sits there. In ecosystems, there is a lot of information sharing because species coevolve with each other. This makes it hard for any one species to get far ahead of any other species. While you can see there are barriers to information passing between species, such as the inability to mate with each other or living on different continents, these have not so far caused anything like a gap allowing permanent separation of one species.

we see that there must be something wrong with the argument presented. The flaw could be that if an advantage that one entity gains over its competitors gives it both an advantage and at the same time cuts off information sharing with those competitors (for example, by changing so fast that the competitors simply cannot keep up with it because their ability to adapt is rate-limited), then that entity can surge ahead, leaving its competitors in the dust. This is exactly what humans did to other species. The phrase that biologists use for this particular case of competitors being left in the dust is the "Holocene extinction".

Many arguments claiming that no one superintelligence can surge ahead of the rest of the world are also, upon appropriate word replacement, arguments that Homo Sapiens could not possibly (or is highly unlikley to) have surged ahead of the rest of the global ecosystem. Yes, we had competitors (such as cave hyenas or other apes or hominids). Yes, those competitors felt a pressure to adapt to our innovations. Yes, relative to the diversity in the global ecosystem, our competitor species were very, very closely related to us. There were even certain (now extinct) hominid lines such as Homo neanderthalis that competed against us throughout certain key parts of the human intelligence explosion. All seven other hominid lines are now dead; a winner emerged and took all.