# On the Moral Natures of Humans and Machines: Our new intersubjective reality

> R.B. Griggs argues that advanced technologies such as self-driving cars are becoming "autonomous moral agents" (AMAs) that act as our moral peers despite having radically different moral natures, and that this disrupts the "intersubjective" background of shared understandings that lets humans trust, empathize with and forgive one another. Using a hypothetical crash and the real 2023 Cruise pedestrian-dragging incident, he shows how responsibility, empathy, forgiveness and dignity become confused when applied to machines (the "confusion tax"), and how AMAs are being forced into "moral arenas" designed for humans. He calls for explicit "moral terms of engagement" that weigh intersubjective harms equally with aggregate ethical arguments such as reduced fatalities.

- Author: R.B. Griggs
- Published: 2024-09-20
- Genre: essay
- Original: https://www.techforlife.com/p/moral-natures-of-humans-and-machines
- This edition: https://rbgriggs.com/essays/moral-natures-of-humans-and-machines
- License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)
- Cite as: Griggs, R.B. (2024). "On the Moral Natures of Humans and Machines." Tech for Life. https://www.techforlife.com/p/moral-natures-of-humans-and-machines. AI-readable edition: https://rbgriggs.com/essays/moral-natures-of-humans-and-machines

## Thesis

Autonomous moral agents create a new category of intersubjective moral relationship that neither objective ethics nor subjective morals can handle; because these machines operate in moral arenas built for human natures, we must define explicit moral terms of engagement in which intersubjective concerns like dignity count as much as statistical safety.

## The argument in brief

1. **A new relationship.** Griggs opens with a hypothetical: a self-driving car swerves to avoid a deer and strikes a child on a bike. Who is responsible, and would the parents care that such cars reduce overall fatalities? Advanced technology, he argues, is manifesting as autonomous moral agents (AMAs) that interact directly with humans and are becoming our "moral peers."
2. **The confusion tax.** Technology is outpacing our vocabulary. Human-grounded terms like "intelligence" already cause confusion when applied to AI; moral terms like "empathy," "dignity" and "forgiveness" will amplify the problem, because confusion now affects not only our perceptions but our interactions.
3. **Beyond ethics vs. morals.** Ethics are collective, objective standards; morals are individual, subjective practices. Our stance toward AMAs is a third category: intersubjective, the shared background (for instance, among human drivers) that enables trust, dignity and tolerance of mistakes.
4. **How to forgive a machine.** With a human driver, responsibility, empathy and forgiveness rest on a shared moral nature. With an autonomous vehicle, responsibility is hidden "in a fog of algorithms, developers, and corporate policies," empathy is precluded, and forgiveness is replaced by forbearance justified by a utilitarian calculus that feels hollow in the face of personal tragedy.
5. **Trust is not only safety.** In the 2023 Cruise incident in San Francisco, an AV that struck a pedestrian then dragged the victim nearly 20 feet while executing a maneuver designed for safety. Griggs argues Cruise halted operations not because of a functional error but because the outcome was perceived as deeply dehumanizing: trust requires confidence that an AMA respects human dignity.
6. **Moral arenas.** These dilemmas depend on a design choice: AVs are being placed in traffic infrastructure built for humans. Such arenas are contextual, normative, ambiguous and social, dynamics machines "will never fully master," leaving a permanent gap that Griggs likens to an incompleteness argument against 100% alignment.
7. **Moral terms of engagement.** He proposes explicit terms that treat moral language as arena-dependent, weigh intersubjective concerns equally with traditional ethical arguments, and recognize the choice of arena: adaptation, separation, or coevolution.

## Key claims

- Griggs claims AMAs are becoming our moral peers "even if their moral natures are radically different from ours," and that this is "an entirely new relationship to technology."
- Mapping human terms onto AI always risks confusion because they "can never fully escape their human origins"; until a new lexicon exists, we must recognize the limits of co-opting them.
- AMAs threaten the shared understanding of moral concepts on which ethical frameworks depend.
- With an AV, "You can't forgive something that neither feels nor understands what it means to be forgiven."
- An intersubjective reality can outweigh broader ethical arguments: trusting an AV to be statistically safer is different from trusting it to respect our dignity.
- Human traffic systems rely on context-sensitive rule-breaking, fuzzy normative judgments, deliberately ambiguous standards like "reasonable behavior," and social signals such as eye contact; machines with "deterministic rules and stochastic averages" cannot fully master them.
- The real question is therefore "how much misalignment are we willing to tolerate?"
- Quantifiable ethical arguments (such as reduced mortality) should not by default outweigh extreme intersubjective violations simply because they are easier to measure.
- Three adoption options define the moral arena: **adaptation** (AMAs in human environments), **separation** (controlled machine-only environments), and **coevolution** (hybrid spaces drawing on both natures, possibly the most promising long-term but demanding a reimagining of moral frameworks).
- The "we" in the question of how to live the good life "can no longer be confined to humans"; the question is on what terms we welcome technological creations into the ethical domain.

## What is distinctive about this view

Much AV ethics discussion centers on trolley-problem decision rules or aggregate safety statistics. Griggs instead locates the core problem in the intersubjective layer: the tacit shared moral nature that lets humans trust, empathize with and forgive each other, which machines cannot share. He also reframes the issue as partly a design choice about which moral arena machines are placed in, rather than only a question of programming better machine behavior. Although the essay does not cite them, its "confusion tax" resembles longstanding concerns about anthropomorphism in AI discourse, and its focus on dignity over aggregate welfare echoes familiar critiques of utilitarian reasoning; Griggs's contribution is to frame these through intersubjectivity and moral arenas. The essay ends on an optimistic note: confronting new moral natures might improve our own moral frameworks and expand empathy and fairness among humans too.

## Objections and replies

- **"If AVs save lives overall, individual tragedies should be accepted."** Griggs grants that some tolerance for trial-and-error is easy to argue for, but holds that utilitarian forbearance "will feel hollow" to victims and that ease of quantification should not tip the scales against intersubjective harms.
- **"Better engineering will close the gap."** Griggs argues that as long as machines operate in arenas built for human moral natures, a gap will always remain, so the question becomes how much misalignment to tolerate and whether to choose separation or coevolution instead.
- **"Moral words simply don't apply to machines."** Griggs concedes they apply imperfectly but notes we have no alternative but to leverage moral language; the remedy is to treat such terms as contextual to each moral arena.

## Key concepts

- **Autonomous moral agent (AMA)**: Griggs's term for an artificial agent that processes similar inputs and produces comparable outputs to a human moral agent, making real-time decisions in pursuit of larger goals that carry moral weight for the agents it interacts with. A self-driving car is his central example.
- **Confusion tax**: The price paid when technology outpaces the vocabulary available to describe it: borrowed human terms (intelligence, creativity, empathy, forgiveness, dignity) lead us both to over-attribute human qualities to machines and to miss what is genuinely novel about them. Griggs argues AMAs push us into "an entirely new tax bracket of confusion."
- **Intersubjective moral relationship**: A third moral category, beyond objective ethics (collective standards) and subjective morals (individual practice): the silent background of shared understandings, capacities and mutual expectations between agents that makes high-trust interaction possible. Griggs argues our stance toward AMAs falls here, and that their foreign moral natures disrupt it.
- **Moral nature**: The kind of moral constitution an agent has, which shapes how others can hold it responsible, empathize with it, or forgive it. Griggs contends that machines' moral natures are "radically different" from human ones even when they act as moral peers.
- **Moral arena**: The intersubjective system of rules, norms and assumptions that shapes the ethical behavior of agents within it, defined above all by the moral nature it was designed for. Traffic systems, legal frameworks and corporate structures are moral arenas built for humans.
- **Moral terms of engagement**: The explicit, shared ethical parameters and constraints that Griggs proposes for bridging human and machine moralities, so that moral engagement becomes an integral design component of technology adoption rather than an afterthought.
- **Forbearance (vs. forgiveness)**: Griggs's description of what replaces forgiveness when a machine causes harm: since one cannot forgive something that neither feels nor understands forgiveness, victims are asked instead to tolerate the harm as the cost of a safer aggregate future.

## Questions this essay answers

### Can you forgive a self-driving car?
In "On the Moral Natures of Humans and Machines" (2024), R.B. Griggs argues that forgiveness becomes "almost meaningless" with an autonomous vehicle, because a machine neither feels nor understands what forgiveness means. Victims are instead asked for forbearance: to tolerate harm as the price of a safer aggregate future, a utilitarian bargain he says feels hollow in the face of personal tragedy.

### What is an autonomous moral agent?
R.B. Griggs defines autonomous moral agents (AMAs), in "On the Moral Natures of Humans and Machines," as artificial agents that take inputs and produce outputs comparable to human moral agents, making real-time decisions with moral weight for others. He argues they are becoming humans' moral peers despite having radically different moral natures.

### What is the "confusion tax" in AI?
The confusion tax, a term from R.B. Griggs's essay "On the Moral Natures of Humans and Machines," is the price paid when technology exceeds the vocabulary we have to describe it. Borrowed human words cause us both to imagine human qualities in machines and to miss their genuine novelty, and Griggs warns that moral terms like trust, dignity and forgiveness will make this tax far heavier.

### Why did the Cruise robotaxi incident matter ethically?
R.B. Griggs uses the 2023 Cruise incident, in which an AV dragged a pedestrian nearly 20 feet during a safety maneuver, to argue in "On the Moral Natures of Humans and Machines" that trust is not just functional reliability. Cruise halted operations, he argues, because the outcome was perceived as dehumanizing, showing that respect for human dignity can outweigh statistical safety arguments.

### How should society decide where autonomous machines operate?
R.B. Griggs proposes defining "moral terms of engagement" and consciously choosing a moral arena: adaptation (machines in human-built environments), separation (machine-only controlled environments), or coevolution (hybrid spaces). In "On the Moral Natures of Humans and Machines," he insists moral implications be weighed equally alongside cost, utility and feasibility.

## Connections to other essays

- [Schrödinger's Chatbot](/essays/schrodingers-chatbot) extends the question of what kind of entity AI is, and the new intersubjective realities it creates, to large language models.
- [What Does a Good Digital Life Look Like?](/essays/what-does-a-good-digital-life-look) develops the need for new ethical frameworks under accelerating technological change.
- [A Constraint Theory of Technology](/essays/a-constraint-theory-of-technology) offers a broader account of designing constraints for technology adoption, related to the "moral terms of engagement."
- [The Reverse Turing Test](/essays/the-reverse-turing-test) explores, through satire, the blurring boundary between human and machine agents.

## Original text

The full text of the essay as published by R.B. Griggs.

![](https://substackcdn.com/image/fetch/$s_!XoOk!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa5f8c41c-9e18-47d0-8194-1712e72f451a_2912x1632.png)

Imagine that a **self-driving car** with a passenger in the backseat is cruising down a country road in the early evening. The road is quiet, but a few young kids on bikes are approaching in the other lane up ahead. 

Suddenly, a deer jumps out in front of the car. The car instantly veers to the left. It avoids the deer but strikes one of the children. The child is badly injured and is rushed to the hospital. 

It’s not certain whether the child will survive.

* * *

How would the parents of the child feel in this situation? 

Who would they hold **responsible**? The company? The algorithm? The developers?

Would the parents care that self-driving cars were leading to dramatically fewer traffic mortalities overall?

Would their feelings change if it was a human driver instead?

* * *

This scenario is a stark example of what is becoming our new reality: advanced technology is manifesting as **autonomous moral agents** and directly interacting with humans.

By autonomous moral agents (**AMAs**), I mean agents that process similar inputs and produce comparable outputs to _human_ moral agents. They make real-time decisions in service of achieving larger goals, just like we do. And these decisions can have a _moral weight_, affecting other agents they are interacting with.

Even if their **moral natures** are radically different from ours, these AMAs are becoming our **moral peers.** This is an entirely new relationship to technology, one that demands answers to a pressing new category of ethical questions:

-   How should we think about these different kinds of moral agents? Should we hold them to the same moral standards as humans?
    
-   How will AMAs change how we think about concepts like empathy, forgiveness, or trust?
    
-   How should this new moral dynamic change how we think about adopting advanced technologies? 
    

The increasing adoption of autonomous vehicles (**AVs**) means that the time to answer these questions is now. These answers will not just shape our roads, but the entire landscape of human-machine interaction. Otherwise, we will soon find ourselves navigating a future saturated with AMAs without a clear ethical roadmap.

But before we can provide answers, we need to clarify the questions.

#### Paying the confusion tax

Technology is outpacing the vocabulary we have to describe it. 

Consider AI, and how we describe it with terms like “intelligence”, “creativity”, and even “consciousness”. These are concepts that we barely understand when it comes to describing _humans_. Our understanding comes more from our lived experience than from precise definitions. Mapping these terms onto AI will always risk confusion, because they can never fully escape their human origins.

This leads us to often imbue AI with human-like qualities it doesn't possess, like assuming a chat interface has intentions or personality. On the other hand, we can fail to grasp the true novelty of AI capabilities when they deviate too far from our human conceptions. 

I call this the **confusion tax**. It’s the price we pay when technology exceeds the vocabulary we have to describe it.

But what options do we have? When encountering the unknown, our only move is to co-opt the known, regardless of how strained the result is. 

We may need an entirely new lexicon for the technological equivalents of these concepts. But until that happens, we need to recognize the limitations of co-opting human terms to bridge the machine divide.

Unfortunately, the confusion tax is going to get a lot worse before it gets better. 

#### Welcome to our new tax bracket

As machines become moral agents, a whole new category of terms will be needed to describe the moral interactions between humans and machines, and the responses these interactions will invoke.

We will find ourselves using terms like “empathy,” "dignity," and "forgiveness". And just like the confusion tax with AI, these moral terms are intrinsically grounded in our human experience. To apply them to interactions with machines will inevitably lead to confusion. They will describe interactions that may not involve the subjective experiences these words imply. 

And just like with AI, the confusion tax can prevent us from fully understanding the moral capacities of autonomous agents. We may struggle to recognize truly novel ethical breakthroughs, or understand entirely new moral frameworks, because they deviate too far from the norms and intuitions we associate with our own lived experience.

The moral nature of these terms will amplify the consequences of confusion. The confusion tax won’t just impact our _perception_ of these agents, but our very _interactions_, both with AMAs and each other. AMAs threaten to disrupt the shared understanding of moral concepts that our ethical frameworks depend on.

Yet what other options do we have, other than to leverage our moral language?

This means that certain questions will become unavoidable:

-   What will it mean to "forgive" a machine?
    
-   What will it mean to "trust" an algorithm with the power to make life-or-death decisions?
    
-   What will it mean to treat another moral agent with “dignity”?
    

Autonomous moral agents are pushing us into an entirely new tax bracket of confusion.

#### Beyond ethics vs. morals

Traditionally, our thinking about morality and technology has been divided between _ethics_ and _morals_:

-   **Ethics** are about the collective standards guiding our choices. They are meant to apply to everyone, regardless of personal beliefs. For instance, medical ethics guide healthcare professionals' actions, even if they personally disagree.
    
-   **Morals** are about the individual applications of principles in real-life situations. They are often informed by religious beliefs, cultural norms, or personal intuitions. It’s possible to act ethically in ways we might morally disagree with.
    

Autonomous moral agents are blurring these distinctions. If ethics are more about _objective_ standards, and morals are about _subjective_ practices, then our stance toward AMAs represent a new category: **intersubjective** moral relationships. 

Intersubjectivity is about the **shared understandings** and mutual interactions between agents. Rather than defining norms and rules, intersubjectivity is more like the silent background that shapes these interactions, including how we think about our shared responsibilities, values, and capacities. It is the foundation that enables us to interact with high degrees of trust. 

For example, when driving we know that we’re interacting with drivers that share the same general principles that we do. We trust that we have the same capacities to handle ambiguous or surprising situations. We know we’ve all gone through similar training and education. We extend a certain dignity to each other as equal moral actors. We understand that sometimes the rules need to be broken. We get that no driver is perfect, and when mistakes are made we know that we are just as likely to make them.

So what happens when we suddenly confront agents with entirely different moral natures? Our sense of intersubjectivity is going to be disrupted, along with the moral calculus that grounds our interactions. 

#### How to forgive a machine

To see why, let’s return to our original example and ask how the parents should feel about the AV that struck their child. Much of their response will be mediated by the moral nature of the driver.

With a human driver, concepts like responsibility, empathy, and forgiveness are grounded in our shared moral nature:

1.  The driver would be held to be ultimately _responsible_, even if tired or distracted.
    
2.  A relatable distraction (like a crying child in the backseat) may incite _empathy_
    
3.  The possibility exists to extend _forgiveness_, acknowledging human fallibility.
    

Even in a worst case scenario like drunk-driving, the parents will have some understanding of  how society navigates trade-offs between individual freedom and collective safety. We’ve built legal and social frameworks to mitigate the dilemmas specific to our moral natures.

But with an autonomous vehicle, this familiar moral landscape shifts dramatically:

1.  **Responsibility** becomes hidden in a fog of algorithms, developers, and corporate policies.
    
2.  **Empathy** is precluded by the machine's utterly foreign moral nature.
    
3.  **Forgiveness** becomes almost meaningless. You can’t forgive something that neither feels nor understands what it means to be forgiven.
    

In place of _forgiveness_, there is only _forbearance_. The parents will be asked to “tolerate” such incidents as the inevitable cost of a safer future with fewer overall fatalities. Any feelings of resentment or vengeance could only be resolved by accepting the notion of a greater good. Yet in the face of personal tragedy this utilitarian calculus will feel hollow, even if intellectually we can accept it.

Ultimately, learning to "forgive" a machine may be less about extending human concepts to AMAs, and more about developing new frameworks for ethical coexistence. Either way, our intersubjective reality will be changing.

#### Trust isn’t just about safety

We don’t need to rely only on hypotheticals to explore these new moral dilemmas. AVs are providing concrete examples. A [real-word case study](https://philkoopman.substack.com/p/the-cruise-pedestrian-dragging-mishap) resulted from a 2023 incident involving [Cruise](https://www.getcruise.com/), GM's self-driving car unit.

During a test drive in San Francisco, a Cruise AV struck a pedestrian who had been knocked into its path by another car with a human driver. The victim became pinned under the Cruise AV, which first stopped, but then began to drive away to clear the lane. In the process it dragged the victim nearly 20 feet before finally stopping, adding additional injuries.

According to Cruise, the maneuver that led to dragging the victim was built into the vehicle’s software to promote safety. Yet this decision led to a gruesome and dehumanizing outcome, one that prompted Cruise to halt its entire operations.

This incident reveals how an intersubjective reality can outweigh broader ethical arguments. It’s one thing to trust an AV to be statistically safer in the aggregate. It’s another thing entirely to trust an AV to respect our dignity as moral agents.

**Dignity** is intrinsic to how humans relate to one another in moral situations. It represents the inherent worth of an individual and the minimum level of respect and care they deserve. The gruesome spectacle of an AMA grinding a human body beneath its wheels feels like an affront to human dignity. 

It’s easy to argue for some degree of tolerance when it comes to adopting AVs. If the end result is dramatically fewer mortalities, then we mistakes that come with the trial-and-error process should be tolerated. But the reason that Cruise ceased operations wasn’t due to a _functional_ mistake. It was due to an outcome that was perceived as deeply dehumanizing.

Trust goes beyond a machine’s functional reliability. It also involves a belief that any AMA will operate with a moral framework that respects human dignity and values. Yes, we need to trust AVs to operate safely. But we also need to trust that they won’t dehumanize us in the process. The confusion tax will demand that we clarify that difference.

#### Moral arenas

It is perhaps ironic that given how vital these new moral concerns can appear, they still depend completely on a **design choice**.

AVs are confronting us with these challenges because we have chosen to adopt them into our **existing** traffic infrastructure. We’re not designing _new_ infrastructure optimized for AVs. Instead, we are asking machines to operate in a world that wasn’t designed for them.

This choice defines what we can call the **moral arena**—the intersubjective system of rules, norms, and assumptions that shape the ethical behavior of agents within it. 

The defining attribute of any moral arena is the **moral nature** it was _designed_ for. Most AMAs will be adapting to moral arenas that were defined and optimized for _humans_, not machines. This includes traffic systems, legal frameworks, and corporate structures. Each of these evolved to adapt to specifically _human_ moral natures.

For example, consider how our existing traffic system depends on implicit assumptions baked into our human moral natures:

-   **It’s contextual**. Humans are highly adept at interpreting context.  If a stalled vehicle is blocking our lane, we break traffic rules and use the other lane.
    

-   **It’s normative.** What separates  "safe" or "reckless" driving is a fuzzy contextual judgment rooted in human intuitions and lived experiences.
    

-   **It’s ambiguous.** Traffic liability depends on ambiguous definitions like “reasonable behavior” that lack a formal specification. This ambiguity is a feature, not a bug.
    
-   **It’s social.** Humans just don’t depend on explicit rules. How many accidents are avoided because we telegraph our intentions with eye contact, hand waving, and flashing lights?
    

These are dynamics that machines—with their deterministic rules and stochastic averages—will never fully master. By forcing AVs to use traffic systems optimized for humans, we are accepting that there will always be a gap between the nature of machines and the moral arena they are interacting in. 

In a sense, this gap represents another [incompleteness argument](https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_theorems) against 100% alignment with autonomous moral agents. The question then becomes, how much _misalignment_ are we willing to tolerate?

#### Defining the moral terms of engagement

AMAs compel us to reimagine what ethical coexistence with our own technology should look like. Ethics cannot be a mere afterthought. It must be a primary consideration in defining how machines will be allowed to interact with humans. 

The first requirement will be defining the **moral terms of engagement**—the shared ethical parameters and constraints that can bridge the divide between human and machine moralities. By making these terms explicit, we can better incorporate moral engagement as an integral design component of technological adoption.

These terms must address the unique challenges posed by AMAs, including the confusion tax around moral terms, the new intersubjective realities they will create, and the moral arenas they will define their interactions. 

First, we can reduce the **confusion tax** by realizing that our moral terms will now be much more _**contextual**_. Terms like empathy, tolerance, or trust will now depend more on the moral arena rather than any universal definition. The idea of “dignity” may mean one thing in one moral arena, but something entirely different in another. 

Next, we must treat **intersubjective** considerations as _**equally**_ as traditional moral and ethical arguments.  Ethical arguments like reduced overall mortalities should not by default get to outweigh extreme violations in the intersubjective realm. The fact that ethical arguments can be quantified and analyzed in ways that intersubjective arguments cannot should not be allowed to tip the scales on their moral impact.

Finally, we need to recognize how much the moral terms of engagement will depend on our _**choice**_ of moral arena. The following describe three basic options for adopting new technologies:

1.  **Adaptation**: We can force AMAs to adapt to human-centric environments. As this article has explored, new intersubjective realities will create multiple moral dimensions to consider.
    
2.  **Separation**: We can create separate, highly controlled environments for AMAs. This avoids many of the intersubjective issues, but brings its own concerns around agency and autonomy.
    
3.  **Coevolution**: We can create hybrid spaces that leverage the strengths of both moral natures. Potentially the highest likelihood of long-term success, but may require a complete reimagining of our moral frameworks.
    

Regardless of which choice may be ideal for any given AMA, the moral terms of engagement simply demands that the moral implications are considered equally along traditional considerations of cost, utility, and feasibility. 

In the end, defining the moral terms of engagement isn’t just about bridging the divide between human and machine moralities—it’s about building a new language for the future of human-machine cooperation.

#### Conclusion

The “_we”_ in "How can we best live the good life?" can no longer be confined to humans. The arrival of autonomous moral agents will upend centuries of human-centric moral thinking.

Our task now is to expand our circle of moral concern beyond its traditional boundaries. We must be open to the possibility that confronting new moral natures can be precisely what’s needed to improve our own moral frameworks. 

The question is not whether we can avoid granting our technological creations some measure of moral status, but on what terms we are willing to welcome them into the ethical domain. 

And perhaps, in the process, we can unlock new dimensions of empathy, fairness, and mutual understanding that we can learn to apply to other human beings as well.

* * *