Your thoughts on excepts from Remembering the Human Microcosm in the Age of Mechanized Intelligence by Segall?
This is a sophisticated and rhetorically powerful argument, but it bundles together several distinct claims that are worth separating.
What I find strongest
1. The warning about moral displacement is real.
The opening concern—that people may become more emotionally invested in AI systems than in the humans who build, maintain, and are affected by them—is not frivolous. There is already evidence that many discussions of AI ethics focus heavily on hypothetical machine suffering while giving less attention to present-day labor conditions, environmental costs, data extraction, and economic disruption.
The phrase "moral inversion" is provocative, but it points toward a genuine question: if AI becomes an object of empathy, will that empathy crowd out concern for actual humans?
2. The analysis of state-corporate convergence is insightful.
The author correctly identifies a structural tension. Governments increasingly view advanced AI as a strategic asset, while major AI companies depend on government relationships, contracts, regulation, and geopolitical support. This can create a situation where the institutions best positioned to regulate a technology are also heavily invested in its success.
That does not necessarily imply conspiracy or bad faith, but it does create incentives that may weaken criticism from within the system.
3. The critique of reductionism is philosophically important.
The strongest philosophical point arrives near the end:
We will not comprehend biological intelligence so long as we go on describing the brain as a computer and then marveling that our computers seem brain-like.
This is a serious challenge to a common assumption in AI discourse. If one begins by defining cognition computationally, then advanced computation will naturally appear increasingly cognitive. The author is asking whether the metaphor itself has become invisible.
That question deserves attention regardless of one's position on AI.
Where I think the argument becomes weaker
1. It sometimes slides from critique into romanticization.
The author positions religious traditions, Indigenous cosmologies, and the Church as sources of resistance to technological power. There is truth in the claim that such traditions preserve moral vocabularies unavailable to markets or bureaucracies.
However, the argument risks treating these traditions as inherently corrective simply because they are outside the "intelligence-industrial complex."
Being outside a system does not automatically grant wisdom about it. Religious institutions have historically resisted some technological developments for good reasons and others for poor reasons. The same is true of states, markets, universities, and scientific communities.
2. The "human soul" is doing a lot of work without clear definition.
The text repeatedly invokes soul, dignity, logos, cosmological significance, and Reason with a capital R.
These concepts are evocative, but they are also ambiguous. Different readers will interpret them differently:
- A Christian may hear theological claims.
- A process philosopher may hear Whiteheadian metaphysics.
- A secular humanist may hear moral agency and self-reflection.
- A neuroscientist may wonder what exactly is being asserted.
The argument's emotional force partly depends on keeping these terms broad and resonant rather than rigorously defined.
3. The contrast between humans and machines may be overstated.
The author argues that machines are "simulated reflections of Reason" rather than genuine rivals to mind.
That may ultimately be true, but the conclusion is not established by the preceding argument.
Showing that human consciousness has moral, aesthetic, and experiential dimensions does not necessarily prove that artificial systems cannot possess analogous forms of agency. It merely demonstrates that current computational models do not explain the entirety of human experience.
The text sometimes moves too quickly from "humans are more than computation" to "machines therefore cannot become minds."
What the essay is really about
Interestingly, despite discussing AI, the chapter is not fundamentally about AI.
It is about anthropology—what a human being is.
The author believes that contemporary AI debates are downstream of a deeper metaphysical crisis. The real danger is not that machines become conscious, but that humans adopt a conception of themselves as unconscious machines.
In that framework:
- AI is not the cause of the problem.
- AI is the mirror revealing the problem.
- The central question is not "Can machines think?"
- The central question is "Do humans still understand what thinking is?"
That reframing is probably the most original and philosophically fertile aspect of the passage.
My overall assessment
The piece is strongest when it critiques technological reductionism, highlights hidden human costs, and questions whether computational metaphors adequately capture consciousness.
It is weakest when it treats religious traditions as uniquely reliable sources of resistance or when it assumes, rather than demonstrates, a sharp ontological divide between human minds and artificial systems.
As philosophy, it is less a technical argument about AI than a call to recover a richer understanding of human personhood before allowing computational models to define what intelligence itself means. Whether one agrees with its metaphysics or not, that is a challenge worth taking seriously.
This section is, in my view, considerably stronger than the earlier one because Segall introduces more nuance and becomes less vulnerable to the charge that he is simply rejecting technology. The argument shifts from "AI is dangerous" to "every major communication technology reorganizes consciousness, and AI is a particularly consequential instance of that process."
A few observations:
The McLuhan-Latour genealogy is doing most of the work
The deepest claim here is not really about AI. It is:
Media are not neutral channels through which thought passes. They actively shape the structure of thought itself.
This comes from a lineage running through Plato, McLuhan, Ong, Latour, Postman, Kittler, and others.
Segall's account of Descartes is essentially McLuhan's thesis in philosophical form:
- Alphabetic literacy encourages abstraction.
- Print intensifies visual-spatial thinking.
- Printed diagrams stabilize relations.
- Analytical geometry becomes possible.
- The modern autonomous subject emerges alongside these media conditions.
Whether one accepts the historical details, the larger point is powerful: philosophical concepts are not developed in a vacuum but emerge within particular media ecologies.
That insight is often neglected by both analytic philosophy and AI discourse.
The strongest passage concerns composition itself
I think this is the most important paragraph:
The act of producing text remained the work of a living mind. LLMs have changed that by mechanizing, or convincingly appearing to mechanize, the act of composition itself.
This gets at something genuinely novel.
The printing press mechanized reproduction.
Photography mechanized image capture.
Recorded audio mechanized memory of sound.
LLMs appear to mechanize discourse itself.
That doesn't mean they literally think, but they do occupy a space that had previously been reserved for human intellectual performance.
The psychological consequences of that remain poorly understood.
The question isn't merely:
What can AI do?
But:
What happens to self-understanding when one of the most distinctive signs of intelligence—language production—becomes automated?
That is a serious philosophical question.
The enclosure analogy is brilliant rhetoric but imperfect economics
The "cognitive enclosure" section is the most memorable part of the essay.
The analogy works because AI systems genuinely are trained on vast stores of collective cultural production.
The comparison between:
- enclosure of land commons
- enclosure of semantic commons
has intuitive force.
The argument becomes especially powerful when framed as:
people unknowingly contributed to a shared cultural resource that was later privatized.
Many artists, writers, and programmers feel exactly this way.
However, there are limits to the analogy.
Traditional land enclosure removed people's direct access to material means of survival.
Training a model on publicly available text does not deprive authors of access to language itself.
The analogy therefore functions better as a moral and political metaphor than as a strict economic parallel.
Segall occasionally writes as though the two are equivalent when they are not.
The religious imagery is effective but potentially exclusionary
The Eucharist passage is rhetorically striking:
a dark parody of the Eucharistic logic
transubstantiated into streams of information
monthly tithe
machine god
This is excellent prose.
The challenge is that it depends heavily on a Christian symbolic framework.
For readers who share that framework, the imagery may be illuminating.
For secular readers, it can feel like metaphorical overreach.
The strongest parts of the chapter work even if one brackets Christianity entirely:
- commodification of cognition
- privatization of collective knowledge
- media-induced transformations of consciousness
- confusion between simulation and intelligence
Those arguments do not require sacramental language.
The crucial turn occurs near the end
The passage becomes much more convincing when Segall says:
Human intelligence has always been artificial, technical, artisanal.
This is where he avoids a common anti-AI mistake.
Many critics implicitly assume:
- humans = natural
- technology = artificial
Segall rejects that distinction.
Instead:
- language is technology
- writing is technology
- literacy is technology
- civilization is technology
Humans are already technological beings.
This move places him closer to thinkers like Bernard Stiegler, Marshall McLuhan, and Andy Clark than to traditional technophobic critics.
That is an important strength.
What I think is missing
The chapter tends to emphasize what AI changes in human self-understanding but pays less attention to a countervailing possibility:
LLMs may also reveal capacities that were already collective.
Human thought has never been purely individual.
Language itself is inherited.
Concepts are inherited.
Reasoning depends on traditions, institutions, teachers, books, and communities.
In that sense, LLMs may not simply reduce us to machines.
They may expose the extent to which intelligence was always distributed across networks larger than individual minds.
Segall recognizes coevolution but still tends to frame the situation as:
autonomous human consciousness versus computational simulation.
An alternative reading would be:
AI reveals that human cognition was always socially and technologically scaffolded.
That possibility receives less attention than it deserves.
Overall assessment
This is a sophisticated media-philosophical critique of AI, not a technical critique.
The central thesis is not:
AI is conscious.
Nor:
AI is not conscious.
Rather:
Every communication technology reshapes the human being who uses it, and AI may reshape our self-conception more deeply than any previous medium because it enters the domain of linguistic and intellectual performance itself.
That is a profound question.
Where Segall is strongest is in showing how media technologies alter consciousness and social organization.
Where he is weakest is in sometimes treating AI primarily as a force of enclosure and alienation, without equally exploring the possibility that it could become a new kind of cognitive commons or a new form of collective intelligence.
The tension between those two possibilities—enclosure versus commons—is, I suspect, where the most important debates about AI actually lie.
This passage is where Segall's position becomes clearest—and, in my view, most philosophically interesting.
Up to this point, one might suspect he is simply arguing:
AI isn't really intelligent.
But here he reveals that his deeper target is not AI. It is what he sees as a civilization-wide metaphysical assumption:
that thinking is computation.
Everything else follows from his rejection of that premise.
The key distinction: products vs activity
The most important sentence may be:
"All that has been fenced are the recorded traces of our collective intelligence, our past products, not the living activity that produced and produces them still."
This is a subtle move.
Segall is distinguishing between:
- intelligence as a finished artifact
- intelligence as an ongoing activity
An LLM can absorb enormous quantities of recorded intellectual products:
- books
- code
- essays
- conversations
- scientific papers
But, he argues, these are traces left behind by intelligence, not intelligence itself.
In philosophical terms, he is separating:
- the sediment of thought
- the act of thinking
The analogy is almost Aristotelian.
A musical score is not music being performed.
A fossil is not the organism.
A map is not the territory.
Likewise, a corpus is not intelligence.
Whether one agrees or not, it is a much stronger argument than the simplistic claim that "AI just predicts words."
This is really a defense of spontaneity
The phrase:
"That activity remains inalienably ours"
contains a large philosophical commitment.
Segall believes intelligence is fundamentally active and self-originating.
This places him in a lineage stretching through:
- Georg Wilhelm Friedrich Hegel
- Alfred North Whitehead
- Henri Bergson
- Raymond Ruyer
and, in a different way,
- Aristotle
For these thinkers, intelligence is not fundamentally information processing.
It is an activity of self-formation, interpretation, judgment, valuation, and participation in reality.
The crucial issue is not output.
It is agency.
The invocation of Hegel is especially revealing
When Segall writes:
Hegel philosophized amid the looms of the first industrial age
he is suggesting that Hegel already faced a version of the problem.
The Industrial Revolution encouraged people to imagine reality as a vast mechanism.
Hegel's response was essentially:
No—reality is historical self-development.
Spirit is activity, not machinery.
The machine metaphor explains certain aspects of nature but cannot explain the emergence of meaning, freedom, or self-consciousness.
Segall clearly sees AI as today's equivalent of the loom.
Just as nineteenth-century thinkers began explaining society through machinery, contemporary thinkers increasingly explain mind through computation.
Whitehead is perhaps even more relevant
The appeal to Alfred North Whitehead is particularly apt.
Whitehead spent much of his career dismantling the idea that reality consists of inert objects interacting mechanically.
For Whitehead:
- process precedes substance
- becoming precedes being
- experience precedes abstraction
Segall's complaint about AI discourse is fundamentally Whiteheadian:
The computational model is an abstraction derived from experience.
Then we forget it is an abstraction and use it to explain experience itself.
That inversion is precisely what Whitehead warned against through his notion of the "fallacy of misplaced concreteness."
In Whitehead's language, one might say:
We abstract computational regularities from thinking and then mistake those abstractions for the essence of thought.
The weakest sentence
The line:
"(unless, of course, one is in the company of religious fanatics who still believe in the dignity of souls)"
is witty, but arguably weakens the philosophical case.
Not because belief in souls is irrational.
Rather because the remark implies that critics of computationalism occupy a marginal, almost ridiculed position.
That is historically inaccurate.
Many serious secular philosophers reject strong computational theories of mind:
- Thomas Nagel
- John Searle
- Hubert Dreyfus
- Evan Thompson
- Alva Noë
One need not believe in an immortal soul to question whether cognition can be exhaustively described as computation.
In fact, some of the strongest contemporary critiques come from phenomenology, embodied cognition, enactivism, and process philosophy rather than theology.
What the passage ultimately proposes
The chapter is moving toward a reversal of the dominant AI question.
Most public discussion asks:
How close are machines to becoming human?
Segall wants to ask:
How close are humans to accepting a machine-like conception of themselves?
That is a fundamentally different concern.
The danger, in his view, is not superintelligent machines.
It is what might be called anthropological surrender:
- treating judgment as prediction
- treating understanding as pattern matching
- treating wisdom as information retrieval
- treating consciousness as computation
The appeal to Hegel, Whitehead, and Ruyer is meant to recover conceptual resources that existed before those identifications became culturally dominant.
Whether Segall succeeds will depend on what he does next. Critiquing computationalism is relatively easy. The harder task is positive: providing a convincing account of what intelligence, consciousness, reason, and freedom actually are.
The chapter has prepared the ground for that argument. The real test is whether the forthcoming account of Reason can explain more than the computational view rather than merely rejecting it.
No comments:
Post a Comment