One of the byproducts of our media culture is a surfeit of talking heads specializing in hindsight. It is rare to have an expert risk his reputation and invite the ridicule of colleagues by foretelling an unpopular future; rarer still to have an acclaimed technophile incite controversy and dissension by voicing public concern about technology’s drift into dangerous waters. But that’s exactly what Bill Joy did. Joy is the co-founder and chief scientist of Sun Microsystems. He is responsible for the creation of numerous advanced microprocessor and Internet technologies, which makes him an unlikely candidate for spontaneous Ludditehood. Nevertheless, Joy willingly assumed the unpopular role of speed bump on the technological expressway.

Joy is concerned that predicted developments in three promising scientific fields—nanotechnology, robotics, and genetic engineering—are likely to create dystopian results. “There is,” Joy acknowledges, “no profit in publicizing the dangers.” So why did he take the time to detail them in a thoughtful, if lengthy, 24-page article published in the April edition of Wired? Because, Joy concedes, he can no longer hide his fear, and what Bill Joy fears is the extinction of the human race.

Joy’s predictions were sensational enough to earn him his 15 minutes of fame. Whether actual or portended, the media is ever eager to exploit disasters of biblical proportions. So for a few fleeting moments, he was diligently interviewed, invited to participate in quickly assembled panel discussions, and solicited by prestigious universities to debate the probable impacts of emerging technologies. Then, of course, the survival of humankind was upstaged by such heavyweight concerns as the permanent residence of Elian Gonzalez, and the issue was quickly forgotten.

I was curious about Joy’s apprehensions because a month before the publication of his article, I had written a piece that appeared in the June issue of MC titled “Leeches, Cyborgs, and Nanobots.” In it, I explored related issues, in particular the disturbing implications of melding humans with machines. Notably unsettling was Raymond Kurzweil’s prediction that technology would soon allow human consciousness to be uploaded to a neural computer, resulting in silicon-based pseudo-immortality. Kurzweil, heady with optimism, boasted that the evolution of technology brings us “closer to becoming like God.” To my layman’s mind, amplifying the historically disastrous quest for Godlike attributes seemed a risky aspiration. I was surprised to discover that, to Joy’s technocratic mind, the propagation of unrestrained technology has a high potential not merely of being risky, but fatal.

Joy’s epiphany, by coincidence, also came while reading Kurzweil. In The Age of Spiritual Machines: When Computers Exceed Human Intelligence, Joy came across a disturbing passage detailing a time in the future when work would be done primarily by intelligent machines. Two possible scenarios were described. In the first scenario, the interwoven network of life-sustaining machines becomes so complex as to be unmanageable by humans, and humanity drifts into a state of dependency in which “it would have no practical choice but to accept all of the machines’ decisions.” In the second scenario, the machines necessary for survival are controlled by a small minority of elites. “Because human work will no longer be necessary,” the scenario postulates, “the masses will be superfluous, a useless burden on the system.” Depending on the degree of ruthlessness exhibited by the elites, the masses might be “biologically or psychologically engineered...reduced to the status of domestic animals,” or even exterminated.

The prediction was disquieting, but Joy was further unnerved when he discovered that Kurzweil had not penned the words but was quoting Theodore Kaczynski, the villainous Unabomber. Being in intellectual sync with the Unabomber would rattle any thoughtful person, but rather than dismiss it out of hand, Joy paused to examine the dark side of emerging technologies and concluded that Kaczynski was not far off the mark.

The common catastrophic attribute shared by robots, genetically engineered organisms, and extremely miniaturized devices such as nanobots, is amplification. “They can self-replicate,” Joy cautions; and unrestricted duplication, whether triggered by accident or design, has the potential to do staggering harm.

Runaway replication haunts the nightmare scenarios that nanotechnology devotees call “the gray goo problem.” Given that evolution favors superior genetics, suppose some omnivorous, biomechanical designer bacteria created to devour toxic waste should get loose and begin aggressively replicating and competing with real bacteria. The result could be an invisible plague spread by wind and water, like an influenza but multiplying exponentially with no need of a host or a carrier. It is not beyond the realm of possibility, experts say, that it could reduce the biosphere to dust within a matter of days.

Disaster need not depend on mishap or misfortune. “Nanotechnology,” Joy writes, “has clear military and terrorist uses,” and thus is likely to be deliberately misused. It will be within our purview, Joy warns, to customize devices that will be essentially undetectable and capable of selectively exterminating “only a certain geographical area or a group of people who are genetically distinct.” Once established, these technologies, unlike atomic, chemical, or biological weapons programs, “will not require large facilities or rare raw materials. Knowledge alone will enable the use of them.” Thus, Joy believes, weapons of mass destruction will be replaced by the prospect of “knowledge-enabled mass destruction.” Devices with catastrophic potential will be commercially manufactured and widely disbursed under pressures of competition and profit, and thus available not only to governments, but to small groups and extreme individuals. “I think,” says Joy, “it is no exaggeration to say we are on the cusp of the future perfection of extreme evil.”

Ever since Mary Shelley raised Frankenstein’s monster, humanity has shared an industrial nightmare that machines, as Edward Tenner put it, will some day pass “from stubbornness to rebellion.” Indeed, as Tenner writes in his enlightening book Why Things Bite Back: Technology and the Revenge of Unintended Consequences, “wherever we turn we face the ironic unintended consequences of mechanical, chemical, biological, and medical ingenuity—revenge effects, they might be called.”

Think of revenge effects as the problems caused by technological solutions. They can be both big and small and often go unnoticed for decades. Football helmets, for example, were created to prevent head injuries but often cause them because they are used by players as a weapon. Antibiotics are miracle cures for infection, but they also stimulate bacteria to evolve immunity. DDT killed not only mosquitoes but also birds. CFC’s cleaned disk drive assemblies and shredded ozone. Atomic power was harnessed for peaceful use, but at the cost of Chernobyl. PCBs, dioxin, DES, automobiles—the stories are much the same: Each had revenge effects that were unintended and unanticipated.

With regard to emerging technologies, the issue is not that unintended consequences will emerge—the record strongly suggests they will—but that the stakes are getting progressively larger. Tomorrow’s revenge effect will be more difficult to ignore and, if Joy and others are correct, impossible to survive. “This is the first moment in the history of our planet,” Joy writes, “when any species, by its own voluntary actions, has become a danger to itself—as well as to vast numbers of others.”

Given that he sees a serious potential for cataclysmic disaster, Joy is alarmed that “we have been propelled into this new century with no plan, no control, no brakes.” Product liability suits notwithstanding, there is a widely held cultural trance that technology is self-correcting. Consumer blind faith is coupled with what Joy describes as a lack of humility in the providers of technology and a tendency to “overestimate our design abilities.”

And, if the recipe for disaster is not in the design, it is frequently found in the designer. Physicist Freeman Dyson recounts what he calls “the glitter of nuclear weapons.” The ability to control “the energy that fuels the stars” was too seductive, even for those who knew better. Dyson recalled that a “technical arrogance overcame people when they saw what they could do with their minds.” No one said no.

Saying “no,” however, is precisely the solution Joy offers. The emerging technologies, he believes, are too powerful to shield against, and so he concludes that “the only realistic alternative I see is relinquishment: to limit development of the technologies that are too dangerous by limiting our pursuit of certain kinds of knowledge.”

Nor does he exempt himself from responsibility. “I have always believed that making software more reliable...will make the world a safer and better place; if I were to come to believe the opposite, then I would be morally obligated to stop this work. I can now imagine such a day may come.”

Quite a man. Sadly, few share his ethics, I think, and his solution, though arguably sensible, appears impractical given that not even the political, military, and spiritual might of the medieval Roman Catholic Church could indefinitely forestall the quest for knowledge.

Metaphorically, the dilemma we as technology users face is that technology allows us to cut vast tracts of forest, but it does not tell us when to stop cutting. Technology is enabling, not restrictive. Typically, by the time awareness dawns, 95 percent of the old growth forest is already gone. So the question remains: What price progress?

In Inherit the Wind, the dramatization of the 1925 Scopes Monkey Trial, Spencer Tracy, in his memorable portrayal of Clarence Darrow, speaks of the inevitable price of acquiring new knowledge. Yes, he says, we can master flight, “but the birds will lose their wonder, and the clouds will smell of gasoline.”

Clearly, a price is always built into progress. But, today, much more is at stake than the loss of innocence. From the time humans learned to walk upright, we have been engaged in a race between consciousness and disaster. Joy’s contribution to consciousness, both as a technology provider and as a harbinger, should not be overlooked.

Victor Rozek is a freelance writer with 17 years of industry experience and president of Rozek & Daggett Consulting, a firm specializing in employee and organizational development. He can be reached at This email address is being protected from spambots. You need JavaScript enabled to view it..