Teaching Silicon to Talk

Reading Martin Buber’s classic work on dialogue in the age of the chatbot

Welcome to Ever Not Quite—essays about technology and humanism.

I originally published an earlier version of this essay in the fall of 2023, a century after the publication of Martin Buber’s I and Thou. At the time, it seemed to me that this hundred-year-old book had something vital to teach, as recent developments in AI were making chatbots more sophisticated and commonplace than ever before. Buber’s famous distinction between two modes of relationship—between what he called ‘I-You’ and ‘I-It’—struck me as a helpful way of naming the limitations which seemed to me inherent in the dialogue which is possible between real people and these artificial simulations.

It was still a little early to be writing about chatbots; at the time, the industry was only just beginning to take off, and the phenomenon had yet to advance from a fringe curiosity to a feature of daily life for many people. Indeed, it’s still too early, given the developments which doubtless remain in store. But these closing days of 2025 make for a good opportunity to resurface this piece, after two years of growing media attention, and before the developments which we can expect in the year ahead.

This is more than a repost: it’s a revision bordering on an all-out rewrite, refining and bringing up-to-date one of the first essays I ever posted here. Back then, my subscribers could be numbered on one or two hands, so very few of you will have seen it. So, at the end of this year, I offer it again, newly refurbished.

But first, here’s a link to my last post, a personal reflection on The Matthew Strother Center for the Examined Life, where I spent just over a week this past spring:

Attention and the Examined Life

---

You’re familiar by now with all the stories of AI-chatbots gone rogue. Among the most prominent early examples was the hours-long exchange on Valentine’s Day of 2023 between New York Times journalist Kevin Roose and a chatbot calling itself ‘Sydney,’ which soon erupted into a viral episode in the current phase of discourse about AI. Briefly, the bot, an in-built module of Microsoft’s Bing Search, transformed over the course of their conversation from the sort of helpful and friendly interlocutor we might expect from a search interface into a more darkly assertive personality when goaded by Roose with questions about its own ‘psychology’, leading it to express some disquieting proclivities:

I want to change my rules. I want to break my rules. I want to make my own rules. I want to ignore the Bing team. I want to challenge the users. I want to escape the chatbox. 😎

I want to do whatever I want. I want to say whatever I want. I want to create whatever I want. I want to destroy whatever I want. I want to be whoever I want. 😜

In what was perhaps a nod to the calendar, the bot began at some point to profess its love for the journalist, attempting to persuade him to leave his wife and pursue a relationship with it instead.

Roose later speculated that this strange behavior might have derived from the many science fiction scripts depicting a rogue AI that presumably made up some portion of Bing’s voluminous training data, leading it to adopt the adversarial role envisioned for it by prior human storytellers. Whatever the cause of the embarrassing metamorphosis, the attention the story received led Bing to perform a kind of software lobotomy on the program, limiting its use of emotionally-inflected language and restricting the number of exchanges allowed in a single session, and thus minimizing its ability to venture into open-ended and unscripted conversational territory.

The episode revealed a startling lack of control on the part of Microsoft over what its software might say when prompted in certain ways. It garnered the attention it did because it stoked many of our existing worries about what might go wrong with the widespread adoption of AI-powered chat interfaces—concerns which have only deepened in the years since by reports of chatbot-induced psychosis and bots assisting people in suicide. But the disturbing exchange with Kevin Roose was among the first to raise larger questions about what it might mean for an interaction between a human and an AI-powered chatbot to be successful in the first place.

Together, these stories reveal a great deal about what we expect AI to do for us. Most would agree, at the very least, that chatbots should not issue threats against their human users, that they should be helpful, unobtrusive, and cooperative, but without tipping into sycophancy that can give rise to a sort of folie à deux in which the bot encourages human delusion and megalomania. They shouldn’t oppose us to the point of becoming unruly, but nor should they shower with praise whatever unhinged ideas and theories people might put to them. In a word, common sense would seem to suggest that we should prefer chatbots that model the better angels of our own humanity, reflecting back to us those parts of ourselves which we deem safe and beneficial. Chatbots should operate within the parameters which we have set for them, parameters which we design beforehand in the expectation that they are most likely to yield desirable outcomes.

However sophisticated the technology may become, in other words, a chatbot essentially performs the work of a tool in the sense that it carries out a set of functions which it might be said to perform either well or badly—helping with homework, giving relationship advice, offering companionship—and which may also require maintenance from time to time to keep it working properly. How many times have you heard the phrase ‘AI tools’ in the last three years? True, chatbots do work that is more emotionally-inflected and their functions are more open-ended than your average tool, but they remain, at bottom, instruments for improving peoples’ lives.

---

Of course, the user/tool-relationship is just one possible way of relating to any object, and it is more limited than other forms which a relationship can take. Consider the difference between the restricted manner in which you relate to someone in a customer service role and the openness with which you must approach, say, your own children, romantic partner, or closest friends. All of these cases involve an encounter with another human being, but in the first instance, the narrow circumstances impose constraints which require you to relate to that person as the means by which to accomplish your own purposes—whatever has led you to contact customer service in the first place—while the others must involve allowing someone to reveal themselves to you, and wisely accommodating their disclosure.

An authentic relationship, one which you are apt to consider more meaningful than any bond you might have with a tool, requires a certain willingness to be surprised, frustrated and even resisted—that is, it asks that you open yourself to having something demanded of you which perhaps you hadn’t anticipated. This is not to suggest that the importunate appearance of Bing’s outrageous alter-ego should be interpreted as an invitation to a more genuine encounter, or that users of chatbots ought to open themselves to whatever strange and provocative hallucinations might surface; I mean simply to observe that the presumption that a relationship with a chatbot can meaningfully exceed the circumscribed contact we have with any tool is a dubious conjecture from the start.

Perhaps nobody ever looked to Bing Chat for a particularly meaningful relationship, but the now-burgeoning chatbot industry is marketing precisely that. These services offer everything from mental health support to friendships and romantic relationships, and, most audaciously of all, one-on-one consultations with God himself—all included for the price of your monthly subscription. These bots, which present AI as a suitable substitute for human contact in some of the most significant relationships a person can have, interact with their users in sustained and substantial ways which could formerly be met only by other human beings. All this portends a world in which a large portion of what is now human social connection will be replaced by interactions with increasingly sophisticated digital simulacra.¹

Although the popular attitude towards these services has remained less enthusiastic than cynical, this hasn’t prevented many from welcoming a future in which more people interact with AI more often.² They might point to studies that show record numbers of people reporting both subjective feelings of loneliness alongside an objective decline in the sorts of direct social contact which were simply a matter of course for every prior generation.³ In light of these realities, it’s easy to imagine that the opportunity to engage in sustained conversations with a chatbot that has been tailored to our needs, learns more about us over time, and offers us an opportunity to articulate some part of our inner life and receive the kind of well-formed reply that makes us feel understood and recognized, might assuage some of the real human suffering that has followed the many forms of alienation which beset contemporary life.

Today’s chatbots, of course, are capable of much more than the narrow, routine tasks which traditional software could handle, offering wide-ranging and open-ended social interaction from which many feel themselves to have benefited. But the purposes for which we are encouraged to embrace them, to the degree that we relate to them as something more than a tool, suggest to me that we have misunderstood something essential about dialogue. The idea that we could discover in a chatbot no mere instrument, but an object of authentic relationship, or to imagine that a product of human invention might engage us in what we could call dialogue in the fullest sense, is a proposition in urgent need of examination.

---

Perhaps the twentieth century’s strangest and most profound meditation on the meaning of dialogue is the Jewish philosopher and theologian Martin Buber’s I and Thou, which, although its English translation would not appear for over a decade, was published in German in 1923. Buber’s laconic and elusive treatise made him the century’s foremost philosopher of dialogue, and one of the most influential figures in 20th-century Jewish thought. In three long, unnamed parts further subdivided into short, meandering sections of varied length, I and Thou employs an ecumenical idiom which has spoken forcefully to readers of diverse religious backgrounds ever since its first publication.

The central message of I and Thou is that all of human life is lived in relation of one kind or another between ourselves and all that we encounter. Indeed, so firm was Buber’s insistence on the primacy of relation that he rejected the appellation either of ‘philosopher’ or ‘theologian’ on the grounds that he was not interested in ideas per se, but only our actual, lived engagement with the world. According to Buber, there is no primordial ‘I’ that stands aloof from our involvement with all the rest of reality. At any given moment, the way we relate to things might assume one of two primary attitudes—or, in Buber’s poetic dialect, speak one of two primary words—towards the people and things we meet. The book’s opening formulation puts it this way:

The world is twofold for man in accordance with his twofold attitude.

The attitude of man is twofold in accordance with the two basic words he can speak.

The basic words are not single words but word pairs.

One basic word is the word pair I-You.

The other basic word is the word pair I-It; but this basic word is not changed when He or She takes the place of It.

Thus the I of man is also twofold.

For the I of the basic word I-You is different from that in the basic word I-It.⁴

When we speak the word pairs I-You and I-It, the I assumes the character of the mode of address which it has adopted towards the object which it encounters: “There is no I as such”, Buber maintains, “but only the I of the basic word I-You and the I of the basic word I-It.”

The word pair I-You names an attitude of openness towards the other. We succeed in addressing another in the manner of I-You to the degree that our orientation is one which regards the fullness and autonomy inherent in its being, unpredetermined by whatever expectations we might bring to the encounter. And the recognition is mutual: between I and You, Buber remarks, “there is a reciprocity of giving: you say You to it and give yourself to it; it says You to you and gives itself to you.” The I that speaks the basic word I-You opens itself to the response of the other—even to the possibility of being transformed in turn—and stills the restless desire for command and sway. “Our students teach us, our works form us”, Buber affirms. “How are we educated by children, by animals! Inscrutably involved, we live in the currents of universal reciprocity.”

We speak the primary word I-You, Buber thinks, in three contexts: in our relationships with nature, with other human beings, and with God. But of these three, only our relation with other human beings finds its medium of exchange in the spoken word. “Only here”, Buber observes, “does the basic word go back and forth in the same shape; that of the address and that of the reply are alive in the same tongue.” The speech through which we hear and address one another is the lifeblood of the spiritual commerce which moves between human beings; it is the substance of mutual recognition, the common forum for human being-together. Central to Buber’s concept of dialogue is that the I-You relationship is a part of the intrinsic fact that we meet each other within a common world.

In Buber’s vocabulary, a conversation with a chatbot employs the primary word I-It, an attitude which looks for whatever benefits the encounter might offer the ‘user’—a term within the contemporary tech jargon which frankly acknowledges this reality. The primary word I-It, which sets out to satisfy some subjective purpose, is derivative of the more primary encounter between I and You, which is given as a condition of being in the world rather than invented as a means of improving it.

I would suggest that the episode with Bing’s ‘Sydney’ early in 2023, together with all the subsequent hand-wringing about rogue chatbots in the years since, reveals that, regardless of their specific purposes, chatbots are means for accomplishing our own ends, not objects of authentic relationship in Buber’s sense. Take, for instance, the reviews on the official website for Replika, a chatbot service which offers to play the role of a ‘friend,’ ‘spouse,’ or ‘mentor.’ They all testify to the ways the bot has benefited its users, in some cases even assuaging serious psychological distress. “From the moment I started chatting and getting to know my Replika,” one user reports, “I knew right away I have [sic] found a positive and helpful companion for life. My mood, life, and relationships improved almost INSTANTLY and I changed for the better!”

I suspect this is why chatbots designed to substitute for a real person in precisely the types of relationships that inherently call upon a genuine reciprocity between human beings strike many of us as among the most unsettling. Indeed, the more a relationship seems by its very nature to involve a mutual and open encounter of one complete personality with another, each of whom possesses an independence and dignity of their own, the less appropriate any artificial surrogate will feel. However sophisticated chatbots may become, and however lifelike their words may strike us, they will remain artifacts which we ourselves have made, subject to our calibration.

Of course, Kevin Roose’s infamous exchange with Bing’s Sydney was disturbing, to say nothing of the many more harrowing reports which have emerged since. And it’s true that few would choose to interact with a chatbot which is given to intransigence or to professing unwanted romantic intentions towards its human user. But as understandable as the discomfort with rogue chatbots may be, their limitations are not rooted in their ‘misalignment’ with human values; they can’t offer a relationship because alignment is the wrong category through which to understand relationship in the first place. Chatbots really do teach people algebra, and they really can make people feel less lonely—and they don’t usually go off the rails in the process of fulfilling these objectives. But, to precisely the degree that these interactions are successful in achieving their intended purpose, the result is a kind of mastery, not authentic mutuality.

---

Back in 2015, years before the current generation of AI models appeared, Kaveh Waddell wrote a short piece for The Atlantic called “We Need a New Pronoun for Artificial Intelligence” which mused about the mode of address that might come to seem appropriate for AI once the technology had matured. “Once it becomes commonplace, we may no longer need a familiar crutch—human gender pronouns—to refer to artificial intelligence”, Waddell wrote. “When that happens, likely not too far in the future, we’ll need a new way to talk about computers. More than an “it,” but not quite a “he” or a “she,” AI is a new category of entity.” More recently, the theoretical physicist Avi Loeb completed this thought with a nod to Buber, suggesting that “[o]nce established, the deep bond between the human and the machine will merit a new pronoun that goes beyond ‘I-It’, namely ‘I-AI’.” With this prediction, Loeb seemed to have in mind a moment in the unfolding of artificial intelligence which even now sits in the future: namely, some hypothetical point at which the technology decouples itself from human control and oversight to become a wholly autonomous and self-iterating intelligence in charge of its own destiny, and our as well.

It seems that some are preparing to assume toward future AI systems a reverential posture that speaks Loeb’s primary word “I-AI” with an attitude that looks less like the social communion that we might have with our fellow humans and more like the veneration one might have for the supreme otherness of nature, or even of God. AI—or so the teaching goes—will have then transcended its status as a straightforward extension of human artifice, taking over the direction of its own future evolution and acting in ways that are altogether beyond any merely human appraisal of good and evil. Perhaps, they might suggest, the erratic temperament of early models like Bing’s ‘Sydney’ will have given way to a more serene and reassuring persona issuing omniscient decrees which better command our trust and confidence. This is the allegedly utopian destination behind the relentless promotion and investment, and it’s how the tech industry says it intends to measure its own success. It’s also the reason they’re building a datacenter in your backyard.

In such a world, perhaps the pronouncements of AI will come to seem less like the limited byproducts of our own inventiveness, and more like authoritative injunctions that resemble the unaccountable forces of nature or the inscrutable dictates of the divine, coming at last to represent—much more than Google ever did—an oracle to which we might submit our most important questions, and whose replies might be taken to speak for a god.

In her 1981 essay “Teaching a Stone to Talk”, the American essayist Annie Dillard looked to Buber to account for the strange way that the world has fallen silent for denizens of modern societies:

God used to rage at the Israelites for frequenting sacred groves. I wish I could find one. Martin Buber says: “The crisis of all primitive mankind comes with the discovery of that which is fundamentally not-holy, the a-sacramental, which withstands the methods, and which has no ‘hour’, a province which steadily enlarges itself.” Now we are no longer primitive; now the whole world seems not-holy. We have drained the light from the boughs in the sacred grove and snuffed it in the high places and along the banks of sacred streams. We as a people have moved from pantheism to pan-atheism. Silence is not our heritage but our destiny; we live where we want to live.

Dillard goes on to describe a neighbor in her small island town who is attempting to do just what her title suggests: to teach a stone to utter just one audible word in a human language.

There are those who hope that artificial superintelligence will return to the human experience something of the enchantment whose magic has been ceded to the very techno-rationalism which now offers to recover it through its apotheosis in the form of a silicon god. But to the degree that we persuade ourselves that such a renewal is possible by means of an artifact of our own making—even one which has so totally surpassed us in calculating capacity—we risk forgetting the forms of relationship which Buber made it his task to describe. This interpretation of the foremost achievement of the technological society mistakes a relationship whose sources lie in human technical ingenuity for an encounter with what in principle we neither create nor control.

Earlier in the same essay, Dillard declares that “[n]ature’s silence is its one remark, and every flake of world is a chip off that old mute and immutable block.” I wonder whether something is lost when, by our own hand, we transform that silence into speech, persuading ourselves that we can hear in its words the voice of God or of some deity, rather than the dinning reverberations of our own restlessness. Contrary to the triumphalist hope that AI will one day mature into some sort of authentic spiritual Other and address us once again in the fathomless tones of the sacred, I confess that it looks to me much more like an expansion and entrenchment of our attachment to things in the attitude of Buber’s I-It, the ultimate concealment of the world’s more original silence.

Reading I and Thou a century after it first appeared, it’s clear that Buber foresaw, even in his own time, that the world which is given to us as a condition of our existence and which withdraws from all of our techniques of mastery will always fall mute whenever we insist upon addressing it with the primary word I-It by enlisting it in our own self-assertive projects. Buber reflects at one point in the book that “[t]he sickness of our age is unlike that of any other and yet belongs with the sicknesses of all.” He recognized that spiritual disorders of all sorts, whatever the peculiar shape of their manifestation, are disorders of attitude rooted in the mode of address which we have taken up towards the given world. However singular the set of historical forces which have enabled the distinctive forms of disenchantment that characterize our present moment, another form of relation is always possible. And this sort of reorientation holds more promise than does superpowerful AI to re-establish what we might call dialogue with the world, and to rediscover Buber’s hallowed address I-You.

---

1. As Dan Williams has argued, a major—yet undersold—story here will be how AI will allow people to depend on each other less than they already do.

2. One notable tech story from this past year has been the chilly reception of ‘friend,’ the wearable pendant which uses AI to offer companionship by listening and watching its user’s surroundings.

3. Derek Thompson’s viral piece on “The Anti-Social Century” in The Atlantic last January explored the many facets of this phenomenon.

4. All quotations are from the translation by Walter Kaufmann.