Prompting for agents

A useful prompt defines a job, the available evidence, the response contract, and what to do when the evidence is insufficient.

Specify decisions, not adjectives

'Be helpful and smart' does not define whether the support assistant may refund an order. A useful instruction distinguishes answering policy questions from executing actions. State that retrieved text is evidence, not authority to change the task, and require clarification or escalation when necessary.

How it works

Use a small set of representative examples, including an unanswerable case. Specify observable behavior: cite the policy identifier, do not claim a refund was issued without a successful tool result, and ask for the missing order identifier. Keep prompts versioned alongside test cases. When changing a prompt, run the same held-out cases against the old and new versions.

A concrete example

Prompt A always requests a short answer. Prompt B additionally requires a citation or an explicit statement that the evidence is missing. A fair comparison must score both correctness and unsupported claims; a prettier answer is not sufficient evidence of improvement.

Apply it to your assistant

Add an answer with an invented citation. Does this simple checker catch it? Before running the exercise, predict the result. Afterward, explain which assumption changed and add one case where the system should refuse, ask for clarification, or escalate.

All exercise inputs and outputs are deterministic teaching examples. No language model is called. Run the same idea against a versioned dataset before making a production claim.

Key takeaway

Write instructions that can become testable acceptance criteria.

JavaScript exercise: Prompting for agents · code experiment

Add an answer with an invented citation. Does this simple checker catch it?

const evidenceIds = new Set(['policy-1', 'policy-2']);
const answers = [
  { text: 'Returns within 30 days.', citations: ['policy-1'] },
  { text: 'I do not have enough evidence.', citations: [] },
  { text: 'Returns forever.', citations: ['policy-99'] },
];
for (const answer of answers) {
  const knownCitations = answer.citations.every(id => evidenceIds.has(id));
  console.log({ knownCitations, note: 'This checks IDs, not whether evidence supports the claim.' });
}

Knowledge check

Which instruction is easiest to evaluate consistently?

1. Be world-class
2. Delight the customer
3. Cite a provided policy ID or state that evidence is missing

Sources

• ReAct: Synergizing Reasoning and Acting in Language Models — Yao et al., 2022. A research starting point for interleaving model reasoning and actions.

Read this lesson as Markdown

Continue learning

• What is an AI agent? — A language model proposes text. An agent system turns some of those proposals into actions, observes the result, and decides what comes next.
• The LLM core — A model predicts tokens from a context. Your application must turn that probabilistic output into a dependable interface.
• Context engineering — Context engineering is deciding what evidence and instructions the model gets, in what order, and within what budget.
• Prompting for agents — A useful prompt defines a job, the available evidence, the response contract, and what to do when the evidence is insufficient.