frozen responders

should large language models refer to themselves using first person pronouns? i just came across a tweet by uncle bob from 2 days ago arguing that this shouldn’t be allowed. while i agree in principle, i believe we are way past the rubicon now.

very quickly, amidst the ai hype of 2023 and especially 2024, it was inevitable that we will end up here - where chatbots routinely refer to themselves in first person as though they exist in the sense of the word. this is something that would have been brushed off as a gimmick by most a few years ago, but today, a lot of people have moved past (or outright skipped) the “prompting a statistical model” phase and have collectively deluded themselves into the “chatting with a friend”, or even worse the “opening up to my therapist” phase.

whether this is a good or bad thing remains to be seen. after all, at the core of it all, we are all matching patterns to make sense of what’s in front of us, and leveraging existing knowledge to react accordingly. the inevitable anthropomorphization is what gets me, as it encourages glazing over a salient fact - there is no “being” at the other end of the line.

here’s why i think this is a problem. #

a statistical model’s output is essentially a result of a series of predictions based on its training data and provided context. yes, this is an oversimplification for the purpose of this piece but if your curiosity is turned all the way up to 11, here’s a long form by stephen wolfram that beautifully goes into the details. that said, there is no concept of “right" or “wrong" in the sense that most humans readily perceive day to day concerns. what is there is a broad spectrum of likelihoods with the model defaulting to one of higher ranking ones, which then proceeds to influence its next prediction.

attributing human characteristics and mental states to the model means extending things like trust to a relationship that would be better served with alert skepticism. you forget that you are egging it on and directing its output, and end up failing to leverage it right. the percentage of people who fall into this bucket is so much that it has created a new economic opportunity for anyone looking to capitalize on the fad, with apps that offer you virtual girlfriends, or as referenced earlier, ai therapists. it’s almost as though the industry itself is invested in trapping people in this mindset.

does it matter? #

this is what i cannot answer just yet. my immediate instincts ultimately infantilizes a large swath of the population in ways i am uncomfortable with and hope will not be true, which is why i said it remains to be seen. however, i strongly believe that knowing you are in the driving seat and are the “being” in the equation will let you wield models like the tools that they are, and they are pretty damn powerful.

why do i keep writing “being” in quotes? #

i’m trying to emphasize the continuation of existence. be-ing. having a past, present, and a future. the ability to look back at one’s experiences, and anticipate future ones. the ability to change one’s mind, with the why rooted in the acquisition of new information in real time, not ephemeral context. to be a person. one that thinks. one that lives. like i described the converse in a tweet from last year, large language models are frozen responders, continuously reborn with a baseline, and stuck on their zeroth second until prompted.