Words without the fog
A short definition to keep moving, and a precise one when you want to go deeper. Jargon has to earn its place.
A family of techniques that let computer systems perform prediction, classification, generation, or decision tasks.
The term covers many different approaches. This project focuses mainly on models whose parameters are adjusted from data.
A mathematical system fitted from data to transform inputs into outputs.
Related lesson: Model, chatbot, agent: what is the difference?A large language model designed to process sequences of tokens.
A generative LLM can produce text, but it is not, by itself, a complete chatbot or application.
A text unit associated with a numerical identifier so a model can process it.
A token may be a whole word, part of a word, punctuation, or another unit. It is not always a word.
A system that turns text into a sequence of tokens and identifiers.
Related lesson: What is a token?The finite list of units a tokenizer can map directly to identifiers.
Related lesson: What is a token?A value between 0 and 1 used here to weight a candidate token.
It depends on the model, context, and generation settings. It is not the probability that a statement is true.
The set of probabilities assigned to the possibilities under consideration.
Related lesson: How does an LLM generate a response?A positive setting that changes how concentrated the token distribution is.
Lower values usually concentrate probability on leading candidates; higher values flatten it. Temperature adds no knowledge.
Randomly selecting a token while taking its probability into account.
Related lesson: How does an LLM generate a response?The rules used to turn model scores into a sequence of tokens.
Related lesson: Why can two answers be different?An initial state used to produce a reproducible sequence of pseudo-random numbers.
Related lesson: Why can two answers be different?The tokens a model can take into account when producing the continuation.
Related lesson: What is the context window for?Using a trained model to calculate an output from an input.
Related lesson: How does an LLM generate a response?A plausible output that is false, unsupported, or incompatible with the requested source.
Related lesson: Why can an LLM hallucinate?The limit on the token sequence available for one generation.
Related lesson: What is the context window for?A mechanism that computes weighted relationships between positions in a sequence.
Related lesson: What is the context window for?An application that organizes a dialogue between a person and an automated system.
Related lesson: Model, chatbot, agent: what is the difference?A system that chooses steps and can act with tools in pursuit of a goal.
Related lesson: Model, chatbot, agent: what is the difference?An external function an application can call to retrieve data, calculate, or act.
Related lesson: Model, chatbot, agent: what is the difference?