A Simple Mental Model for AI Techniques

One of the most exhausting things about joining the AI field after years in software development is organizing the terminology. You get to hear about RAG, fine-tuning, few-shot examples, n-shot examples, tool use, chain-of-thought…like they are all very different things with different objectives. They are not.

The only thing all these techniques do is modify two pieces of the LLM: the weights (training time) and the input (inference time).

Training time is the equivalent of the development phase of a software product. You do it before releasing it to customers. In software development, this means writing code, tests, validating with real users, CI/CD, etc. In the LLM world, this means feeding a model with lots of data and messing with its configurations to modify the model's weights and get better results.

Inference time, on the other hand, is like production. The LLM does not change anymore. When you send a message to ChatGPT, that’s inference time. Just like in compiled software, the only thing that can be changed at this time is the input.

Here’s one way to think about it: an LLM is just a function.

# An LLM's output is a function of its input and weights
output = function(input, weights)

Every technique you’ve ever heard of — RAG, fine-tuning, chain-of-thought, DPO, RLHF, n-shot prompting, etc — modifies one of those two things:

• Inference time techniques change the input. The model’s weights stay exactly the same. You’re changing what goes into the LLM.

• Training time techniques change the weights. You’re modifying the LLM itself.

I. The training time bucket

These techniques modify the model’s internal weights. They are more expensive and require that you host your own version of the model after the optimization. Here are some examples:

Pre-training: the massive initial training run on internet-scale data. This is where the weights are created in the first place. Done by companies like OpenAI and Anthropic.

Fine-tuning: further training on specific data to specialize the model. The weights shift to reflect the new data. Techniques like RLHF, QLoRA, and DPO are all variations of this idea. They all modify the model's weights, just in different ways and for different purposes

The important distinction here is that these techniques produce a different model. After training, the weights are different from what they were before.

II. The inference time bucket

These techniques modify the input. That’s all you’re doing, modifying an input. Not fancy at all. That’s why they tend to be way cheaper, simpler to implement and therefore more popular.

Prompt engineering: writing better instructions. The weights don't change. You're just giving the model better input to work with.

RAG (Retrieval-Augmented Generation): fetching data from a database (documents) and stuffing it into the prompt. The model is the same. You just add more data to the input.

Few-shot examples: showing the model examples of what you want. Again, you're modifying the prompt, not the model.

Chain-of-thought: asking the model to reason step by step in the prompt.

Tool use: giving the model access to external functions. And the tool's output? It goes right back into the prompt.

Notice the pattern: in all of these, all you’re doing is changing the input (prompt).

III. Why this matters

When you’re choosing a technique, you can immediately narrow your options. Don’t have training infrastructure? Then you’re working in the inference time bucket. When you read a paper or announcement, you can instantly place it. “They improved results using chain-of-thought”, inference time. “They trained on 10x more code”, training time. When something breaks, you know where to look. Is the issue with what the model knows (weights → training time) or how you’re asking (input → inference time)?

Most developers working with LLMs today are doing inference time work: writing prompts, setting up RAG pipelines, agentic tool-use patterns, etc. And that’s where the leverage is for most applications.

The model is like a car engine. You can build a better engine (training time), or you can learn to drive masterfully (inference time).