Tag Archives: game theory

Ray Kurzweil Revisited/Reanimated

Yesterday, I received the most interesting and cogent response yet to my Tinker Point idea. It came from my friend Rob, who is himself a biologist. For those who don’t follow the comments on my posts, here is his comment in its entirety, because I like it. My thoughts on his remarks follow.

Interesting ideas here but the evolutionary reasoning is a little false. There are already examples in nature of convergent individuals that are very successful. So successful in fact that they have all but given up individuality for group success. The family Physaliidae is the best example of a society that maintains both a division of labor as well as an absolute requirement of cooperation.

But higher eusocial organisms also fit the bill, such as hymenopteran and isopteran societies that grow into vast numbers of of individuals. Even mammals have eusocial organiosms in the naked mole rats of East Africa.

I would not be so quick to imagine an evolutionary disadvantage to being susceptible to mental manipulation. Organisms that “allow” themselves to be domesticated do very well, much better than their wild type counter parts. Domestic chickens have gone global and outshine jungle fowl to a ridiculous extant. All horses are now domesticated. Dogs, corn, wheat, sheep, pigs, all have allowed themselves to be under profound control and in so doing have far outstripped their wild ancestors in evolutionary success.

I think what your argument does cover is that there are some very dark solutions to a future that includes a Kurzweil Singularity. If you mean that we may lose something of our individuality, or our humanity, then you are right to hold that fear. But game theory indicates not that individuality will be maintained, but rather that choices will be made that enhance a payoff. And every payoff is not the same to everyone in the game. So if something, such as a collective creature of electronic, biological, or amalgam, gets an advantage over a bunch of individuals then game theory and evolution both predict the payoff goes to that “something.” And by the way, evolution does’t care if an advantage is short term, it only cares about who wins right now. That’s why we still have global warming, and that’s why Kurzweil is correct.

These are great points. I agree with most of them.  Rob’s science is solid. And yet I still think Ray Kurzweil is wrong, and that my evolutionary perspective holds up. Why? Because the Tinker Point and the idea of collaborative human aggregation are not at odds with each other.

The idea of the Tinker Point is that no intelligent species gets past the point where it’s easier to tinker with themselves than to not. It says nothing about the pattern of cooperation under which that limit is reached.

I think Rob’s take here is that in a cooperative society in which humans lose a little humanity in order to bind together into a hyper-intelligent whole is one in which humanity is suddenly on the same page, and thus the incentive to destructively tinker goes away.  And while this might not look nice to individual human beings, it still constitutes a singularity. But I don’t think this interpretation holds water.

In nature, patterns of cooperation are always vulnerable to defection.  This is true at every scale, from transposons operating within individual cells, up to bankers cheating entire national economies for profit.

Furthermore, the force that keeps defection in check and enables cooperation to persist is that of group selection. Our cells don’t collapse in revolt every day because cells that revolt lead to non-viable organisms. We aren’t instantly riddled with cancer the moment we’re born because creatures whose cells don’t cooperate never get as far as reproducing. At every level at which agents cooperatively aggregate, there has to be something that the aggregates are competing against in order for the cooperation to be maintained. Bind the entire species into a single hive-mind and your incentive has gone away. Massive social collapse at that point is eventually inevitable.

Now, it’s fair to say that the Singularity as Kurzweil defines it doesn’t necessitate a single hive-mind. Here’s a definition from Wikipedia:

The technological singularity, or simply the singularity, is a theoretical point in time when human technology (and, particularly, technological intelligence) will have so rapidly progressed that, ultimately, a greater-than-human intelligence will emerge, which will “radically change human civilization, and perhaps even human nature itself.”

So a benign Singularity might entail the emergence of a set of collective intelligences, all competing with each other. But this outcome is also suspect. How is this convenient outcome supposed to happen? And in this world of competing hive-minds, what role does intelligence actually play?

If intelligence isn’t critical to their competition, we would expect these hive-minds to end up stupid as they optimized to improve fitness, thus conserving the tinker point. If intelligence is critical to their competition, then the mechanisms by which those hive minds operate is still subject to manipulation, and therefore degradation, and therefore overthrow.

In this case the tinker point problem looks just the way it does for individual humans. These hive-minds are still likely to make mistakes and fuck each other up, just like people, until that’s not an option any more.

My favorite speculative future following from this idea comes from the notion that when a new evolutionary niche becomes available, the ‘first to market’ often dominates. I wouldn’t be surprised if, when mental uploading becomes available, that Ray K is first in line to try it out. And once uploaded, of course, he’d rapidly realize that reproducing himself a trillion times and bolting the door behind him would make for a far more stable world than leaving the uplift process open to others. Hyper-intelligence would be unlikely to leave much room for naive idealism.

Hence, if we do end up with future hive-minds at war with each other over the remnants of human civilization, we shouldn’t be surprised if they all have Ray K’s face.