FORFEITING
THE FUTURE
Powerful new technologies threaten life on Earth and raise moral issues
back
to top
ACCUSTOMED TO living with routine scientific break-throughs, we have
yet tocome to terms with the fact that the most compelling new technologies
- robotics, genetic engineering and nanotechnology - pose a different
kind of threat than the technologies that came before. Specifically, robots,
engineered organisms and nanobots share a dangerous amplifying factor:
they can self-replicate. A bomb is blown up only once, but one altered
gene can become many, and quickly get out of control.
While replication in a computer or a computer network can be a nuisance,
at worst it disables a machine or takes down a network or network service.
But self-replication in the new technologies runs a much greater risk:
a risk of substantial damage in the physical world.
Each of these new technologies also offers untold promise: the vision
of near immortality; genetic engineering that may soon provide treatments,
if not outright cures, for most diseases; and nanotechnology and nanomedicine
which can apparently address yet more ills. Together these technologies
could significantly extend our average life-span and improve the quality
of our lives. Yet, with each of these technologies, a sequence of small,
individually sensible advances leads to an accumulation of great power,
and, concomitantly, great danger.
What was different in the twentieth century? Certainly, the technologies
underlying the weapons of mass destruction - nuclear, biological and chemical
- were and are a powerful and enormous threat. But building nuclear weapons
required, at least for a time, access to both rare raw materials and highly
protected information.
The twenty-first-century technologies, however, are within the grasp
of individuals. They do not require large facilities and rare materials.
Knowledge alone will enable the use of them. Thus, we have the possibility
of not just weapons of mass destruction, but of knowledge-enabled mass
destruction, hugely amplified by the power of self-replication.
It is always hard to see the bigger impact of technology whilst in the
vortex of change, but failing to understand the consequences of our inventions
while we are in the rapture of discovery and innovation seems to be a
common fault of scientists and technologists. We have long been driven
by an overarching desire to know: that is the nature of science's quest,
not stopping to notice that the progress to newer and more powerful technologies
can take on a life of its own.
Because of the recent rapid and radical progress in molecular electronics
and related nanoscale technologies, by 2030 we are likely to be able to
build machines a million times as powerful as the personal computers of
today. As this enormous computing power is combined with the manipulative
advances of the physical sciences and the new, deep understandings in
genetics, enormous transformative power is being unleashed. These combinations
open up the opportunity to redesign the world completely, for better or
worse. The replicating and evolving processes that have been confined
to the natural world are about to become realms of human endeavour.
Given the incredible power of these new technologies, shouldn't we proceed with great caution?
THE DREAM OF ROBOTICS is that intelligent machines can do our work for
us, allowing us lives of leisure, restoring us to Eden. How soon could
such an intelligent robot be built? The coming advances in computing power
seem to make it possible by 2030, and once an intelligent robot exists,
it is only a small step to a robot species - to an intelligent robot that
can make evolved copies of itself.
A second dream of robotics is that we will gradually replace ourselves
with our robotic technology, achieving near immortality by downloading
our consciousnesses. We are beginning to see intimations of this in the
implantation of computer devices into the human body. But if we are downloaded
into our technologies, what are the chances that we will thereafter be
ourselves or even human?
Genetic engineering promises to revolutionize agriculture by increasing
crop yields while reducing the use of pesticides; to create tens of thousands
of novel species of bacteria, plants, viruses and animals; to replace
reproduction, or supplement it, with cloning; to create cures for many
diseases, increasing our life-span. We know with certainty that the profound
changes in the biological sciences are imminent and will challenge all
our notions of what life is.
Technologies such as human cloning have, in particular, raised our awareness of the profound ethical and moral issues we face. If, for example, we were to re-engineer ourselves into several separate and unequal species using the power of genetic engineering, then we would threaten the notion of equality that is the very cornerstone of our democracy.
THE MANY WONDERS of nanotechnology were first imagined by the Nobel-laureate
physicist Richard Feynman in a speech in 1959, in which he described how
manipulation of matter at the atomic level could create a utopian future
of abundance, where just about everything could be made cheaply, and almost
any disease or physical problem could be solved using nanotechnology and
artificial intelligences.
Imagine some of the changes that might take place in a world where we
had molecular-level 'assemblers'. Assemblers could make possible incredibly
low-cost solar power, cure cancer and the common cold by augmentation
of the human immune system, could clean up the environment, create inexpensive
pocket supercomputers, and restore extinct species.
The enabling breakthrough to assemblers seems quite likely within the
next twenty years. Molecular electronics should mature quickly and become
enormously lucrative within this decade, causing a large incremental investment
in all nanotechnologies.
But we can't simply do our science and not worry about the ethical issues.
Unfortunately, as with nuclear technology, it is far easier to create
destructive uses of nanotechnology than constructive ones. Nanotechnology
has clear military and terrorist uses, and you need not be suicidal to
release a massively destructive nanotechnological device: such devices
could be built to be selectively destructive, affecting for example, only
a certain geographical area or a group of people who are genetically distinct.
An immediate consequence of the Faustian bargain in obtaining the great power of nanotechnology is that we run a grave risk - the risk that we might destroy the biosphere on which all life depends. For example, as Eric Drexler explained in his book, Engines of Creation: "'Plants' with 'leaves' no more efficient than today's solar cells could out-compete real plants, crowding the biosphere with an inedible foliage. Tough omnivorous 'bacteria' could out-compete real bacteria; they could spread like pollen in the wind, replicating swiftly and reducing the biosphere to dust in a matter of days.
Dangerous replicators could easily be too rapidly-spreading, tough and small to stop. We have trouble enough controlling viruses and fruit flies. We cannot afford these kinds of accident with self-replicating assemblers."
THESE POSSIBILITIES ARE all undesirable. The only realistic alternative
is relinquishment: to limit development of the technologies that are too
dangerous, by limiting our pursuit of certain kinds of knowledge. Although
humankind inherently 'desires to know', if open access to, and unlimited
development of, knowledge henceforth puts us all in clear danger of extinction,
then common sense demands that we re-examine our reverence for knowledge.
If we could agree, as a species, what we wanted, where we were headed
and why, then we could make our future much less dangerous - then we might
understand what we could and should relinquish. If the course of humanity
could be determined by our collective values, ethics and morals, and if
we had gained more collective wisdom over the past few thousand years,
then a dialogue to this end would be practical, and the incredible powers
that we are about to unleash would not be nearly so troubling.
One would think that we might be driven to such a dialogue by our instinct
for self-preservation. Individuals clearly have this desire, yet as a
species our behaviour seems not to be in our favour. The new Pandora's
boxes of genetics, nanotechnology and robotics are almost open, yet we
seem hardly to have noticed. Ideas can't be put back in a box: unlike
uranium or plutonium, they don't need to be mined and refined; they can
be freely copied. Once they are out, they are out.
Verifying relinquishment will be a difficult problem, but not an unsolvable
one. We are fortunate to have already done a lot of relevant work in the
context of the Biological Weapons Convention and other treaties. Verifying
compliance will also require that scientists, technologists and engineers
adopt a strong code of ethical conduct, resembling the Hippocratic oath,
that they cease and desist from work creating, developing and manufacturing
knowledge-enabled technologies of mass destruction.
Where can we look for a new ethical basis to set our course? We would do well to consider a new book by His Holiness the Dalai Lama called Ethics for the New Millennium. As is perhaps well-known but little-heeded, the Dalai Lama argues that the most important thing is for us to conduct our lives with love and compassion for others, and that our societies need to develop a stronger notion of universal responsibility and of our interdependency. He proposes a standard of positive ethical conduct for individuals and societies and further argues that we must understand what it is that makes people happy, and acknowledge the strong evidence that neither material progress nor the pursuit of the power of knowledge is the key - that there are limits to what science and the scientific pursuit alone can do. o
A full-length version of this article first appeared in the April 2000 edition of Wired magazine.
Bill Joy is co-founder and Chief Scientist of Sun Micro-systems and was Co-Chair of the Presidential Commission on the Future of IT Research.