💭 in your head · nothing to type
Throughout the course you worked with the model from the outside: you phrased the task, held the context, plugged in tools, assembled agents. The model stayed a black box that sometimes answers brilliantly and sometimes confidently makes things up.
Recall the last time that happened. The agent named a function that doesn't exist, invented a link, or "remembered" a fact that never was.
What do you think: is this a glitch that will soon be fixed — or is it how the machine itself is built?
💭 in your head · nothing to type
Keep both options in mind. If it's a glitch, you can just wait for the next version. If it's a property of the design, you'll have to learn to live with it.
In module 09 you already worked with sources, and module 11 covers memory and search. Consider why grounding an answer in a retrieved source works at all.
The answer becomes clear once you see what the model actually learns.
The model doesn't read letters or words. Text is cut into tokens — pieces of words, sometimes whole words, sometimes single characters. Each token maps to a number in the model's vocabulary.
Hence the everyday oddities: the model is bad at counting letters in a word and gets confused by long numbers. It sees chunks, not letters.
Each token number turns into an embedding — a long vector of numbers. Pieces with similar meaning land close together in this space. The same idea powers semantic search in your AI notebook (module 09) and the second brain (module 11).
The core mechanism of modern models is described in the paper "Attention Is All You Need" (Vaswani et al., 2017). The architecture is called the transformer.
The gist of attention: for each token, the model decides which other tokens in the text to look at, and how strongly. In "the cat didn't fit in the box because it was too small," the word "it" should look at "box."