ChatGPT hallucinates. We all know this already. But on Tuesday it seemed like someone slipped on a banana peel at OpenAI headquarters and switched on a fun new experimental chatbot called the Synonym Scrambler.
Actually, ChatGPT was freaking out in many ways yesterday, but one recurring theme was that it would be prompted with a düzgüsel question — typically something involving the tech business or the user’s job — and respond with something flowery to the point of unintelligibility. For instance, according to an X post by architect Sean McGuire, the chatbot advised him at one point to ensure that “sesquipedalian safes are cross-keyed and the consul’s cry from the crow’s nest is met by beatine and wary hares a’twist and at winch in the willow.”
These are words, but ChatGPT seems to have been writing in an extreme version of that style where a ninth grader abuses their thesaurus privileges. “Beatine” is a particularly telling example. I checked the full Oxford English Dictionary and it’s not in there, but Wiktionary says it relates to the theologian Beatus of Liébana, a scholar of the end times who died in the year 800, so maybe “beatine” meant “apocalyptic” at some point in the first millennium CE. Or, judging from how it’s used in dusty old books, maybe it’s just another way of saying “beatific” which one would think is already an obscure enough word. In other words, ChatGPT was giving new meaning to the term “esoteric.”
The chatbot was briefly doing things like this to tons of its users. One Redditor, homtanksreddit, noted that ChatGPT 3.5 — the one available to free users — was apparently unaffected, so the bug may have only affected paying users.
In a number of cases, ChatGPT started by answering the user’s question in clear, contemporary language before devolving into an over-caffeinated version of Martin Prince. One user on Reddit helpfully included a screengrab from ChatGPT that included one prompt used to generate ChatGPT’s madness. In it, the user described the bug ChatGPT was experiencing to ChatGPT, and then asked what such a bug is called. Indeed, ChatGPT’s answer started off well:
There isn’t a commonly used, specific term for when a language model like GPT (Generative Pre- trained Transformer) starts providing output that descends from a clear, relevant response into gibberish or total irrelevance. However, this issue could be described as a type of “output degradation” or a “failure mode” of the LLM (Large Language Model).
Not bad. But then it got weird, noting that “byte-level […] miscreance can lose institutional shading.”
…then it got weirder still:
In real-world application, if you notice an NLP system returning fine commencements that then unravel into lawlessness or written collapse, it may involve jumbled inceptions, affected parts blindness, higher perplexity stoked in modules, or a notably malfunctioned determiner thrust — a multicause sachem, really.
Such behavior is worth documenting for developer autopsy, per se, or cycling in fail-safes, learning the gait, and feed clutch, spanning in relativity so developers might trait and palm to cohere or distend correction.”
So there’s your answer. ChatGPT was clearly suffering from a bad case of “jumbled inceptions” with a bit of “affected parts blindness” and acute “higher perplexity stoked in modules.” Simple as!
Many users wondered on X if ChatGPT was having a stroke:
…and at least one wondered if he was the one having a stroke:
But as of Wednesday morning, I was unable to provoke ChatGPT into generating one of these wild outputs, even when I specifically asked it to drone on as much as possible about a boring topic. So it’s safe to say the situation was temporary.
Early Wednesday the bug page for this issue said the sorun had been identified, but was still being monitored. By late morning, however, the page listed the issue as “resolved.” When asked for comment, an OpenAI PR rep referred Mashable to the general status page for ChatGPT, which just says “All Systems Operational,” as of this writing.
Mashable asked OpenAI to elaborate on what had happened, perhaps in an obscurantist and grandiloquent style, but the request was not immediately granted in the fullness of our unstinting if somewhat caviling journalistic desiderations.
Topics
Artificial Intelligence