Breaking the Spell of LLMs

I. “We Are Not in Traditional Programming Anymore”

“How is this possible?” was the feeling I had when I first watched Copilot autocomplete a file with tests that were, for the most part, correct. At the time I didn’t have the energy to think too much of it. Like most developers working on startups in the early 2020s, I was pressured by deadlines and could not afford the time to understand something purely out of intellectual curiosity.

Indeed, I was guilty of benefiting from AI without thinking too much of it. Treating it like magic. But no more.

We all know the rules of traditional programming. You write the steps, the code executes and there’s an output. The paradigms for this have been many, from structured, to object-oriented, to functional, but it all boils down to control and visibility: you can map the flow of information and understand exactly what is going on within your software. Whether you're a backend or frontend developer, whether you’re working with Rails, Python, C# or COBOL, this is your place in the world.

Machine Learning is something essentially different. Instead of programming the steps directly, you expect to accomplish a task by learning from data how to do it. Think of a SPAM filter. In traditional programming, you would write rules like “should not contain the words ‘lottery’, ‘click here’, ‘prize’, etc". With Machine Learning, you take thousands of emails that were already classified as SPAM or NOT SPAM and feed a model with those examples so that it can figure out which patterns represent SPAM.

Nobody writes the rules. The model learns them from the data and will be able to categorize any new email.

II. But What is a Model, Exactly?

The answer may surprise you: a model is software. It’s code written by research scientists and engineers. Mostly in Python. But unlike the code developers write, it does not contain business rules or logic. It is a mathematical function with adjustable parameters (or weights). This is true from very simple models, like our email example, to complex ones, like LLMs.

An LLM’s default behavior is to predict the next token. From a developer’s point of view, this is a fancy way of saying that an LLM’s job is to autocomplete. If you used VSCode with Copilot a few years ago, you may recall that autocomplete was all it did initially. It would autocomplete code according to the software’s context. More advanced behaviors, like chat and “agent”, only came later.

Machine Learning is the broad field, the idea that computers can learn from data instead of following explicit rules. Neural Networks are one technique within Machine Learning, inspired by how biological neurons connect. Deep Learning is what happens when you stack many layers of neurons, allowing the model to learn increasingly abstract patterns. Transformers are a specific Deep Learning architecture introduced in 2017 by Google researchers. And LLMs are what you get when you take a Transformer and train it on huge amounts of text: billions of pages of books, articles, code, and conversations.

From a software engineer's perspective, you could say a Transformer looks like MVC. It’s the architectural pattern. PyTorch, JAX and TensorFlow are like Rails and Django, the frameworks you use to implement that pattern. And what Anthropic, OpenAI, and Google do is what you’d compare to building an actual application — they take the architecture, implement it using frameworks, make their own design decisions (how many layers, how many parameters, what data to train on, etc), and produce a trained model like Claude, GPT, or Gemini.

Just like Spotify uses Rails or you might use Django on a personal project, OpenAI uses PyTorch and Google uses JAX for developing their models.

III. How to Create a Model (The Big Picture)

For both simple models, like our spam filter, and complex ones like GPT-4 or Claude, the pipeline follows roughly the same steps. Keep in mind that this is a simplified overview. I’m leaving out important concepts like tokens and tokenization, embeddings, loss functions, gradient descent, eval metrics, and many more. The goal here is to get the big picture.

A. Create or choose a model architecture
Spam filter: You pick an existing architecture (for instance, logistic regression) and import it using a library like scikit-learn. One line of code. You're not building the architecture from scratch. You're choosing which one fits your task.

LLM: Anthropic creates its own version of the Transformer architecture. A model of its own. This takes tons of research and experimentation.

B. Perform the training
Spam filter: You feed thousands of emails already labeled as SPAM or NOT SPAM. The model adjusts its parameters until it gets good at telling one from the other. One phase, a few minutes on a laptop. Done

LLMs: Training is split into two phases. First, pre-training: the model is fed massive amounts of text and learns language broadly. After this phase, it can complete sentences, but it can't hold a conversation or follow instructions. This is what is called a base or foundation model. Then comes fine-tuning: the model is trained on examples of conversations, instructions, and human feedback. This is what turns a text-completion model into the Claude or Gemini you actually talk to.

C. Inference (AKA “running the model”)
Spam filter: A new email arrives. It goes through the trained model’s parameters. The model outputs: SPAM or NOT SPAM. Every time Gmail catches a suspicious email, that’s inference. It’s the equivalent of running the code you created.

LLMs: You type “explain tokens to me” and hit enter. Your message goes through billions of trained parameters. The model generates a response, token by token. That’s inference too.

If you want a more concrete example, check Karpathy’s microgpt article where he explains how to build a model with only 200 lines (!) of Python. I’ll sure put it on my study roadmap.

IV. Stop Confusing Models With Products

This may seem trivial, but so many developers make this mistake that it's worth addressing: GPT-4, Claude Opus 4.6 and Gemini 3.1 Pro are models. ChatGPT is not a model. It is a product. It has a UI, memory, can search the web, generate images, read files, etc. LLMs can’t do any of that by themselves. AI products like ChatGPT orchestrate different models and software tools to deliver their user experience.

One could say that in order for the model to be accessible for regular users it needs to be wrapped in software. It’s not literally inside — it’s accessed via API — but you get the idea. What becomes obvious is that models are going to be a fundamental component of software from now on.