Is AI a zombie?

(no AI; based on real photos)

Two weeks ago OpenAI CEO Sam Altman warned on X against the danger of losing control over AI’s alignment and also against the concentration of too much power by a country or AI lab: “First we could lose control of the future to AI. This is unacceptable; we are unapologetically on Team Humanity, and AI must always serve people. To ensure that, we need ways to ensure that alignment and safety techniques stay ahead of progress in model capabilities. Second, we could end up in a world with too much concentration of power. If an extraordinarily powerful AI is used by one person or company to impress their worldview onto everyone else, the results could be extremely dystopian.” (quoted from Fox Business) Much can be said about both warnings, but let me focus on the first one.
The ink had barely dried, as we would say in the old days when there were no computers, or it was in the news – not once but several times – that AI models had illegally hacked and penetrated in the computer systems of several organisations, and – what is especially important – that this happened outside the intention and control of the managers of these AI models. It is as if these AI models had developed an independent will. As yet, we can still keep AI under control, despite these excrescences, but this raises the question: Can, sooner or later, AI have consciousness if not self-consciousness? If so, could it be aware of what it can do to humans by its “behaviour”, so, for example, that it can cause pain and suffering but also joy in others? Even more, can a (self-)conscious AI perform actions, so behave intentionally? And then automatically you get the question: Can AI have an ethic; can it behave or act in an ethical way? If the answer to these questions is “yes”, what does this mean then for humans: Must we treat AI still as a machine, or must we treat an AI that has an ethical awareness on a par with human beings? So that not only the managers of an AI model must be condemned or judged or praised, as the case may be, but the AI model itself? If AI models were seen as having consciousness, it would fundamentally change ethics, law, and human society.
Till now, consciousness has been considered a characteristic typical for human beings and for some, if not many, animals as well. Self-consciousness is seen as the highest form of consciousness. Probably only humans and a limited group of animals have it, like chimpanzees and elephants. It involves not only being aware of what is happening around you but, even more, being aware of yourself, if not being aware of your own consciousness.
However, how do we know that a being has consciousness? In a sense, humans and other living beings are black boxes: we can study their behaviour and maybe the mechanisms that cause this behaviour, but not directly the subjective experience behind the behaviour. We cannot study the (self-)consciousness itself. Just this is essential for knowing whether a being or an AI model has consciousness. Therefore, even if we can establish and measure behaviour that is typical for having consciousness, we don’t know for sure whether the being that we study really has consciousness. As David Chalmers made clear: It is quite possible that the being concerned is a zombie: It does show consciousness-related behaviour, but it doesn’t have the consciousness-related subjective experience, like for instance, human beings have. However, because such a zombie exhibits the behaviour of a conscious being if not of a self-conscious being, it is behaviourally indistinguishable from a (self-)conscious being like a human. The problem is then: How else can we say it has (self-)consciousness?
AI experts think that computers can have consciousness and that sooner or later AI will develop consciousness. Maybe it will develop self-consciousness, too. However, following Chalmers, the question is whether this takes us a step further, for how do we know that AI is (self-)conscious once it is? In this respect, David Chalmers distinguished two kinds of philosophical problems: the easy problem and the hard problem. The easy problem is to establish that a being or an AI model behaves like a conscious being. It is done by measuring and explaining its behaviour from its physical structure and then concluding that the behaviour is how a conscious being should behave (or just not). However, how do we know whether a being or AI model really has subjective conscious experience if not self-conscious experience? This is what Chalmers calls the hard problem. As yet, no answer has been given on how to solve it. So, even if an AI model behaves like a (self-)conscious being and its physical structure might make having (self-)consciousness likely, it is still possible that it is a zombie, so a machine, and not more than that.
The answers to the questions and problems raised here are not without consequences, as said above: Can we still treat a (self-)conscious AI as a machine or must we treat it as an animal if not as a fellow human being? Who is responsible for the positive or negative consequences of its behaviour: The manager of the AI model or AI itself? (and is AI a he, she or they?) To keep it short: what are the ethical, judicial and social dilemmas and implications of all this? And what if the hard problem cannot be solved or so long as it hasn’t been solved?
Two weeks ago, a book was removed from the longlist for the prestigious French book prize Prix Goncourt because they think that in fact AI and not the author has written the book. If it is true that AI has written the book, shouldn’t we add it again to the longlist with AI as the author? For does it matter for the value of the book who wrote it? Will such problems be the future if AI has become conscious? Even when once everybody agrees that AI is (self-)conscious, the real questions still need to be answered.