Science-fiction writer David Brin has it all figured out
The options before us appear to fall into four broad categories:
1. Self-destruction. Immolation or desolation or mass-death. Or ecological suicide. Or social collapse. Name your favorite poison. Followed by a long era when our few successors (if any) look back upon us with envy. For a wonderfully depressing and informative look at this option, see Jared Diamond's Collapse: How Societies Choose to Fail or Succeed. (Note that Diamond restricts himself to ecological disasters that resonate with civilization-failures of the past; thus he only touches on the range of possible catastrophe modes.)
We are used to imagining self-destruction happening as a result of mistakes by ruling elites. But in this article we have explored how it also could happen if society enters an age of universal democratization of the means of destruction — or, as Thomas Friedman puts it, "the super-empowerment of the angry young man" — without accompanying advances in social maturity and general wisdom.
2. Achieve some form of "Positive Singularity" — or at least a phase shift to a higher and more knowledgeable society (one that may have problems of its own that we can't imagine.) Positive singularities would, in general, offer normal human beings every opportunity to participate in spectacular advances, experiencing voluntary, dramatic self-improvement, without anything being compulsory… or too much of a betrayal to the core values of decency we share.
3. Then there is the "Negative Singularity" — a version of self-destruction in which a skyrocket of technological progress does occur, but in ways that members of our generation would find unpalatable. Specific scenarios that fall into this category might include being abused by new, super-intelligent successors (as in Terminator or The Matrix), or simply being "left behind" by super entities that pat us on the head and move on to great things that we can never understand.
Even the softest and most benign version of such a "Negative Singularity" is perceived as loathsome by some perceptive renunciators, like Bill Joy, who take a dour view of the prospect that humans may become a less-than-pinnacle form of life on Planet Earth.
4. Finally, there is the ultimate outcome that is implicit in every renunciation scenario: Retreat into some more traditional form of human society, like those that maintained static sameness under pyramidal hierarchies of control for at least four millennia. One that quashes the technologies that might lead to results 1 or 2 or 3. With four thousand years of experience at this process, hyper-conservative hierarchies could probably manage this agreeable task, if we give them the power. That is, they could do it for a while.
Via Gravity Lens