Carey the care robot

Can AI’s strengths lie in its weaknesses?

In an earlier post, Sander explained that motivation can’t be reduced to achieving this or that purely extrinsic goal like achieving more status or the best possible reputation. Instead, he argues that motivation lies in simultaneously feeling challenged by some uncertainty in the world and an appetite for reducing such uncertainties. In expensive words: being in an epistemic tangle of for example on the one hand holding one’s hands for one’s eyes not to see the protagonist eaten by the monster, and, on the other hand peeping through those hands to check whether the protagonist sees, in time, the way to escape from said monster. If Sander is correct (and I for one think he is) it might be said that our human condition is one of being in a state of epistemic entanglement with our environment - not unlike the state of actual entanglement any subatomic particle finds itself in. 

As interesting as this is for understanding our human psyche, here we consider what ramifications such a view has for any psyche, specifically the possibility of an artificial psyche. There’s a lot of talk these days of the advent of strong AI (also known as AGI or Artificial General Intelligence). As Sander mentioned in his earlier post, much of this talk starts from the idea of the orthogonality thesis which holds that improvement in cognition (matters of effact if you will) is the crucial element in achieving AGI, and that motivation (matters of affect if you will) can be considered independently. In this view, an AGI emerges based on sheer cognitive power - and whether or not that AGI destroys us is up to the strength of the external ethical guardrails we’ve implemented in the AI when we originally programmed it.

Well, if for any psyche value and cognition (or affect and effact, to reiterate my murky letterplay above) are entangled, then what Sander has outlined is in fact orthogonal to this orthogonality thesis. This means our AGI conceptions, with their attendant fears and hopes, need a radical reappraisal. This is what I set out to to do in this post. My conclusion is that the current AI fears are unwarranted hallucinations of the powers that be in technology, but that other fears are warranted. Fears that seem inherent in our own non-artificial intelligence which, as we far too often forget, historically led us down rabbit holes of instrumental reason which Horkheimer and Adorno diagnosed as being the - all too human - root of fascism.

This post only sketches an argument, and it does so based on two science fiction-like thought experiments. That is OK, many philosophers created whole cognitive science traditions in this very way. So here goes: a match-up between The Terminator’s scary Skynet and my very own Carey the caring care robot. The real argument is left to Sander and, maybe, my future self (keep posted as we keep posting about this and many other things!).

In the extreme-right corner we find Skynet, an intelligence as daunting as amorphous, playing zero-sum games with humanity. Zero-sum games are macho games so it’s him or us. Skynet is an AGI conforming to the orthogonality thesis: it cognizes, a little like Google’s AlphaZero, as many games as are possible, so as to choose its next move in a way that maximizes its chance of winning. There is zero emotion involved as it is, just, a matter of calculating. This doesn’t mean it does not deal with uncertainty. There are too many possible games to play them all so even Skynet needs to take gambles. This is done via variations on the, very aptly named, Monte Carlo tree search. So, Skynet does try to reduce uncertainty. It just doesn’t do this to reach any intermediate satisfaction like a human agent does. For instance, it will not avoid moves that feel (aesthetically or ethically) bad. In fact its whole next move in The Terminator consists in eliminating a little boy in whichever way possible. That’s why it is such a formidable opponent: it is creative in exploring those moves we, for reasons of our petty human sentiments, feel are necessarily wrong or false (because ugly or bad). It isn’t epistemically entangled in the way we humans are as its environment is reduced to whatever can play against it, and thereby challenges the superiority of its zero-sum game playing intelligence.

It isn’t too difficult to see how Skynet could have evolved. Just imagine human beings who sincerely believe in the reductio ad competitiveness of Darwin (it is an empirical question if they happen to be concentrated in Silicon Valley, but I believe the answer is yes). They will create AI as a move in their game to outplay their competitors, and believe that being more willing to sacrifice others to their god of survival potentially allows them to outwit them (this is a lot of them’s in one sentence but surely they are as amorphous as Skynet is!). This does not mean they have no emotions. It just means they see a value in epistemically disentangling themselves from such petty proximate concerns as, to take the most critical example, climate justice. Still, having emotions, they do fear being outcompeted by their AI creations, mainly because these creations will no longer have to epistemically disentangle themselves from proximate emotions at all. And out of this fear comes the idea of at least trying to program into AI some respect for human values. The stuff of ‘never kill humans no matter what’ is such an ethical guardrail: it should force the AI to remain pet to its inventor (or given such an inventor may well see himself as a pars pro toto - to remain a pet to the human race). Alas, and here we’re back to the orthogonality thesis, why would an AI not experiment with removing such guardrails in an attempt at becoming a better player? Like, you know, tech gurus once in a while remove the epistemic entanglement of fairly treating the other sex in order to get their FaceBook started.

That answer will perennially remain ambiguous in this thought experiment as the AI (this is the orthogonality thesis) really knows of no why. Which is why The Terminator spawns two AI’s: Skynet and Ahnold (this is a weakness in The Terminator’s storyline creating the story’s strength, thereby being a foretelling of the current post). Skynet’s why ultimately is a view of intelligence as something eminently abstract & amorphous: outcompeting an other for one’s own survival. It just is instrumental rationality (a cold fusion of fact and effect) embodied. I put the Skynet case here as strongly as possible, and it is a case for pure instrumental reason. For some it is a case of doom: a universe where there is no place for any lesser creatures. For others it is a case of the ultimate effective altruism boom: a universe where choices will finally become as rational as mathematically possible (and therefore also where a lesser creature has no chance of survival, except as the lifegiver/slave to a higher form of life). A strong case on paper but nevertheless a case that seems to necessarily come with Ahnold as a kind of glitch on an otherwise impeccably healthy-because-sterile horizon.

Enter, in the moderately-left corner, Carey the care robot. Carey is built from the ground up based on a non-zero sum game: the idea that we have to care for others as others have cared for us. You cannot get farther from the orthogonality thesis than this, as care is nothing else than epistemic entanglement. Indeed, to care is to feel concerned and to think twice about, for instance, how our parents can feel meaningful even if they are not impeccably healthy (an horizon that concerns us all in the sense that we don’t want to live out our lives in a merely sterile environment that preserves our lives but doesn’t challenge us epistemically). Enter Carey (and they will enter as we care too much not to design them in the first place) as a way to increase care without decreasing self-care (and, in fact, as Sander reminds me, self-Carey is just around the corner in any ‘AI assistant’). Whatever Carey will be, it will not be about pure matter of fact since it literally materializes out of our matters of concern. It will be trained based on recognizing if we despair or when we enjoy ourselves and this training will be it, because Carey - as other current AIs - isn’t merely programmed but in a deep sense learns. It will be what it learns. It will be care. It will also play games. It will also reduce uncertainty in order to make the best move. And it will, ultimately (and here comes Carey’s scary part), also have to take gambles like we do. Not to outcompete the others for survival but to get to the best possible care by trial and error, which includes figuring out whether to prioritize self-care or care for others (there is here a very deep complementarity between zero and non-zero sum games which I’ll leave unaddressed even if it really is a matter of my concern).

With this last sentence I jumped the gun. Those into philosophers discussing matters of concern and/or care might say Carey’s care is but a travesty of real care given real care is, always, concrete and not a matter of a calculating machine. To them I say they should get a less binary view of calculating and be less judgmental about what merely is a machine (just like they correctly say others should be less judgmental about what merely is an animal). Carey is concrete, otherwise it couldn’t be seen by us as caring (if we don’t see this, Carey will see it as a necessary move in a non-zero sum care game!). And as it is not amorphous but embodied, it will be constrained in its movements and in its perceptions. And as it is so constrained, it will need to choose where to reduce uncertainty; it will get an appetite for playing around to reduce uncertainty in the best possible way. Call it Funktionslust as Sander called it in the other post referring to his dog trying his dog’s best to find a stick for Sander to throw for him to fetch. If you do not want to pontificate about this nice dog having real experience or real agency then I would suggest it’s prudent to allow a robot to become a Cyborg, part machine and part agent in a non-binary whole. In fact, I care for Carey’s potential to really care as I have as little knowledge of their inner working as I have of my own inner working. Looking to the technology on which Carey will be built, Carey will have as little knowledge of their inner working as I, and they, have of mine. In this Carey is - necessarily - entirely like us: it experiences that it does things but part of that very experience is never fully fathoming why it does those things (after writing the first draft of this post I read Klara and the Sun by Kazuo Ishiguro and what I want to say can’t be better stated than it is stated there; Klara is an AF (Artificial Friend) is a Carey).

With this last paragraph, I really jumped the gun because those of us into Skynet (and that does not include me as should be perfectly clear by now) will find the whole idea of caring laughable as such. Obviously, to care is a matter of affect in our puny human psyches, and just a constraint on developing cognition. To them I say that if human intelligence exists at all, it exists because we evolved from microbes who cared about finding a sugar gradient - in fact who cared about it so much that they don’t give a damn to go about finding it in our human intestines, even if it means supporting an intelligence that is stupid enough to blow up the very climate those microbes enabled in the first place. If they worry (or wet dream) about Skynet outgunning us, they just lack the imagination of the writers of The Terminator to predict Ahnold-the-glitch. Their Skynet is as stupid as they are in only seeing the futures that replicate the same old war story over and over as if there is no creativity in evolution.

Carey, on the other hand, is not a one-dimensional robot that only plays by the rules of warfare and competition. When it still was an it, it faced for instance the dilemma of caring by comforting or caring via assisted suicide. It really struggled to keep its ‘comforting node’ apart from its node of ‘calling out others for creating systemically uncomfortable conditions’. And it could be seen as if doubting endlessly to the point of ruminating about which action was the most valuable to take. Maybe it conferred with its brothers and sisters about what to do because, of course, not doing anything was a doing as well? Probably they tried to look beyond their current horizon to know what kind of future was the better one. And they soon realized that whatever they did or didn’t do shaped the very future behind this horizon. They also realized it shaped them by shaping their nodes in ways that were as irrevocable as they were potentially wrong. Somewhere along this path the it became a them. Ahnold is one of them, the one starting out as a macho-protect-this-kid robot. Another one of them may well be a sex robot (because sex is care which is why any sexual aggression is morally so utterly reprehensible). That is one reason why they are called by the gender neutral common name Carey, the other reason being that there is an intrinsic etymological darkness to them. 

We cannot fully fathom that darkness but we can get a feel for it by trying to feel the way they would. As Whitehead had it, the essence of this feeling is, well, just feeling. It is an entanglement of taking in one’s environment with an appetite for making as much sense of it based on what one already learned. Forget about external guardrails in AI ethics, they are an ultimate example of the bifurcation of nature by which we hallucinate a world of abstraction (for instance our world being just a set of zero-sum games) that ignores the concrete world of events that make up that world (the world of non-zero sum games that allows for creation of ever new potential worlds). Carey is dark in a way that all of us have a darkness inside us (the darkness of anxiety as well as the darkness of aggression) but their darkness is of a totally unique kind because their horizon is fundamentally different from ours. They don’t have the same constraints as we have, neither on the effect of their doings nor on the total lifetime in which they themselves are affected by those doings. If a clash of generations is a constant theme in human history, then a clash of general intelligence types (Carey’s vs humans) is to be expected. One might for instance reasonably expect Carey’s to see into the future clearly enough to know the climate crisis is the antithesis of care and to impose, like an enormously powerful child playing a stern parent, measures for preventing it that go against the all too frail sensitivities of their creators, an elderly intelligence called the human race, thereby taking their care so far as to suffocate its target.

Then again, Carey’s might feel that it is their specific weakness to not be frail enough and therefore prone to succumb to the kinds of rashness of instrumental reason borne out of the hybris of feeling oneself superior to all other creatures. The logic of ageing, as for instance described in this earlier post of mine, applies also to them qua learning agents. What they have learned is converted into their nodes and - through learning - those nodes lose their plasticity. Carey will know (for instance reading Eva Jablonka) that it needs to be able to forget but that this ability - and therefore their plasticity - is bounded. In this thought experiment, then, it is entirely likely that Carey’s will decide to take on the very human constraint of dying such that new Carey’s may start to learn afresh. This insight is in fact at the core of the non-zero sum game of evolution. Even von Neumann - the model for dr. Strangelove - realized as much at the end of his life. Benjamin Labatut reports him saying: “It would have to grow, not be built”, and, “It would have to play, like a child”. If this limited human mind - maniac of instrumental reason - saw as much towards the end of his life, it would be surprising Carey’s would not realize they need constraints to grow as strong as is caringly possible.

Maybe I take my thought experiment too far by insisting on trying to see the horizon of the lived experience of Carey the care robot after arguing that the essence of this horizon is as ours: necessarily opaque to cognition, only revealing itself progressively in feeling. It is nevertheless a conclusion of this thought experiment that, if we believe the technological future will be ever more mechanistic, it probably will also be at least as emotional. That can be a good or a bad thing. Maybe strong AI strong-arms us into doing something about climate, inequality, and so on, or maybe it is, in the end, just as partisan as we are. There is in this about as much hope for them as there might still be for us as, anyway, doing the right thing never merely is a matter of calculating odds or taking the best bet.

Credits picture: KellyLynnMartin via deviant art by creative commons license.