The language problem with large language models

When I launched this newsletter at the beginning of December, it was never my intention to make every commentary about AI. It just seems to have worked out that way. There is just so much to talk about, and as I noted in this space last week, we are dealing with a constantly moving target. This week, I want to talk about the language component of large language models.

It may seem obvious enough, but how you communicate with these models matters, and you’ll learn as you begin using them more often that precision matters, writing skills matter – and that may be the most surprising thing I’ve learned about LLMs.

I know I shouldn’t be surprised that language is an important part of interacting with a model built on language, but it turns out that communicating what you want isn’t as easy as it sounds. We already know how important precision is in human-to-human communication, even if we aren’t always good at it. Think about how many great books, movies or plays are based on a miscommunication that turns tragic or disastrous – just about every Shakespeare play – and that’s because it turns out communicating is hard.

Consider this example from an event in December: I asked several people I met at an event happy hour where they were from. I was thinking about what company they worked at, but I got several answers including the cities where they currently live, where they grew up, and perhaps most hilariously from one man who answered, his mother.

It’s a simple enough question, right? Yet there were a myriad of answers and any one of them could be correct. Now imagine, you aren’t talking to a person standing in front of you with all of the subtleties, context clues and cultural nuances of human communication, but are instead having a conversation with a bot powered by a large language model. Then it gets even trickier. I’ve learned that the more precise your prompt, and the more information you give the model, the higher quality the answer you get. Seems logical enough, but I write for a living. My job is to communicate clearly in writing.

Now consider the fact that many people don’t write well, and suddenly we are being told that in the future writing well won’t matter because LLMs will make that skill obsolete. Evidence suggests, however, that the exact opposite is true. In the future, writing well could matter even more because you have to tell the bot exactly what you want it to do, and if you don’t give it enough information, it’s not going to give you the results you want.

This could become compounded with AI agents, AI systems that autonomously execute tasks, requiring even more precise instructions than chatbots. That’s going to require some expertise about how the process works and, yes, an ability to write well, to communicate that to the agent.

If humans struggle with ambiguity, working with LLMs could amplify that problem. Consider that engineers often trained in math and logic versus narrative writing, may face steeper learning curves. When the process of creating a program could be abstracted to describing what you want, and having the AI create it for you, suddenly software development could be less about math skills, so much as writing. That’s mind blowing really.

Regardless, we have created these models built on language, and in the process we have made the process of communicating even more essential than ever before, and that’s an outcome I didn’t expect.

Photo by ThisisEngineering on Unsplash