Summary and Review by Jack Harris
I first read Computer Power and Human Reason in 1977. It was my third year of teaching at Hobart and William Smith Colleges, and I was in a cohort of faculty teaching a first-year general education course, “The Mechanical Bride,” named after Marshall McLuhan’s book. We were examining thinking, especially the idea of “artificial intelligence.” It is where I first learned to code and to test the boundaries of human and machine learning and thinking, including running the “Turing Test.”
Joseph Weizenbaum had been invited to join the MIT Artificial Intelligence Project, created in 1958 by John McCarthy and Marvin Minsky. It was McCarthy who coined the phrase “artificial intelligence,” an imaginative, fantastical conceptualization of machine learning. McCarthy and Minsky were arrogant and asserted that if you could precisely describe the features of human intelligence, you could create an artificial intelligence that could simulate it. Minsky argued that the human brain’s functions could be reproduced, or even surpassed, by human-made computing machines. Ray Kurzweil further developed this claim in his book The Singularity is Near (1974), arguing that the merger of humans with AI was inevitable and that humans and androids would eventually be indistinguishable. Weizenbaum did not share these views. He rejected the idea that the human brain is metaphorically a computer. He criticized the embrace of “instrumental reason,” the idea that we can apply rational technologies to human problems as a substitute for value-driven judgement.
Weizenbaum released his psychotherapist simulation, Eliza, in 1966 (you can still find it on the Internet). As primitive as it was, many people found the program convincing – hoodwinked might be too strong a word. Like Aldous Huxley in Doors to Perception, Weizenbaum argued that it was impossible for humans to ever fully understand another human being, and even less that machine intelligence could. Anthropologists, sociologists, and historians know that, in the lives of human groups and individuals, historical time and space frame culture, social, and psychological differences, including different meanings, understandings, and even emotions and sensations. Then there is the interiority of our private lives, and the challenge of even understanding our own selves.
Computer Power and Human Reason provides a profound critique of the social, ethical, and philosophical implications of artificial intelligence and the over-reliance on technological solutions. Weizenbaum asserts that there must be ethical boundaries for computing: while computers are powerful tools that excel at calculation, they should never be allowed to make crucial decisions that require human judgment, wisdom, empathy, and moral responsibility. AI lacks the capacity for human judgment, which relies on non-logical factors like experience, intuition, emotion, and culture. A computer can process data, but it cannot genuinely experience or care. The four Sixth Level core differentiators, Mutuality, Ingenuity, Justness, and Intrinsic Motivation, are out of AI’s reach.
For Weizenbaum, “Deciding” is an appropriate AI computational activity, following a set of rules (algorithms) to reach a logical conclusion based on available data, and computers can decide far more efficiently than humans. But “Choosing” is a human activity involving values, wisdom, empathy, and moral responsibility. Unlike deciding, choosing cannot be reduced to an algorithm because it requires an understanding of the human condition and context that a machine can never possess. Thus, any activity requiring genuine empathy, human respect, or a comprehensive understanding of the human should not be delegated to a machine. Our over-reliance on computer power and our misplaced faith in instrumental reason risk devaluing and dehumanizing the human spirit.
