The LLM core

A model predicts tokens from a context. Your application must turn that probabilistic output into a dependable interface.

Prediction is not a database lookup

Tokens are pieces of text, not reliably one word each. A model assigns probabilities to possible next tokens and generates a sequence. Its training does not guarantee access to today's shop policy or the customer's order. Supplying evidence can help; it does not make every generated claim true.

How it works

Separate three concerns: the context you send, the decoding settings, and validation of the result. Temperature changes how a distribution is sampled, but temperature zero is not a promise of identical behavior across providers, hardware, or model versions. Pin the configuration and measure repeated runs when variability matters.

A concrete example

A support classifier must return one of refund, shipping, or other. Free text such as 'probably shipping?' may be understandable to a person but break a downstream router. Validate an enum and treat invalid output as a distinct outcome. Never execute an action solely because generated JSON can be parsed.

Apply it to your assistant

Add a new model output with an unsupported intent and observe the rejection. Before running the exercise, predict the result. Afterward, explain which assumption changed and add one case where the system should refuse, ask for clarification, or escalate.

All exercise inputs and outputs are deterministic teaching examples. No language model is called. Run the same idea against a versioned dataset before making a production claim.

Key takeaway

Use models for proposals, typed contracts for interfaces, and evaluation for observed behavior.

JavaScript exercise: The LLM core · code experiment

Add a new model output with an unsupported intent and observe the rejection.

const outputs = ['{"intent":"shipping"}', '{"intent":"refund"}', '{"intent":"delete"}', 'not json'];
const allowed = new Set(['shipping', 'refund', 'other']);
for (const output of outputs) {
  try {
    const value = JSON.parse(output);
    if (!allowed.has(value.intent)) throw new Error('Invalid intent');
    console.log('Accepted:', value.intent);
  } catch (error) { console.log('Rejected:', error.message); }
}

Knowledge check

A response is valid JSON. What does that establish?

Its claims are accurate
Its tool action is authorized
It can be parsed; schema and semantic checks are still needed

Answer and explanation

It can be parsed; schema and semantic checks are still needed

Syntactic validity says nothing about correctness, authorization, or whether required fields have valid values.

Sources

Attention Is All You Need — Vaswani et al., 2017. The Transformer architecture underlying many language models.

Read this lesson as Markdown

Continue learning

What is an AI agent? — A language model proposes text. An agent system turns some of those proposals into actions, observes the result, and decides what comes next.
The LLM core — A model predicts tokens from a context. Your application must turn that probabilistic output into a dependable interface.
Context engineering — Context engineering is deciding what evidence and instructions the model gets, in what order, and within what budget.
Prompting for agents — A useful prompt defines a job, the available evidence, the response contract, and what to do when the evidence is insufficient.