Prompt Engineering Reference

Based on the dair-ai Prompt Engineering Guide.

When to Apply

• Writing or refining system prompts for Claude features
• Designing few-shot examples or CoT demonstrations
• Building classification prompts with locked SME rubrics
• Structuring RAG retrieval prompts
• Debugging LLM outputs that miss intent
• Instrumenting prompts for eval before shipping

1. Prompt Anatomy

Every effective prompt has four addressable slots:

Omitting output format is the single most common source of inconsistent LLM behaviour.

[SYSTEM / INSTRUCTION]
You are a classification assistant. Given a customer message, return a JSON object
with keys `category` (string) and `confidence` (0–1 float). No prose.

[CONTEXT]
Categories: billing, technical_support, general_inquiry, escalation.

[INPUT]
Customer message: "{{message}}"

[OUTPUT FORMAT]
{"category": "...", "confidence": 0.0}

2. Zero-shot → Few-shot → Chain-of-Thought Progression

Escalate only when the simpler technique fails.

Rule of thumb: start zero-shot, add few-shot if format drifts, add CoT if reasoning is wrong.

3. Chain-of-Thought (CoT)

Standard CoT — include worked examples

Q: A rubric has 4 criteria each scored 1–5. Score: [3, 4, 2, 5]. Is the overall
   score above 14?
A: Sum = 3+4+2+5 = 14. 14 is not above 14. Answer: No.

Q: Score: [4, 4, 3, 5]. Is the overall score above 14?
A: [model continues reasoning]

Zero-shot CoT — append the trigger phrase

Evaluate this response against the SME rubric. Think step by step before giving
your final score.

CoT for locked rubrics (Ask AI / Chakra pattern)

When scoring against a fixed SME rubric, force the model to cite the rubric criterion before assigning a score. This prevents hallucinated justifications.

For each criterion below, quote the relevant excerpt from the response, then
assign a score (1–5). Only after scoring all criteria give the aggregate.

4. Tool-Use / Function-Calling Prompts

Claude's tool-use depends on crisp tool descriptions. Treat each tool description as a mini prompt.

tools = [
    {
        "name": "search_knowledge_base",
        "description": (
            "Search the internal knowledge base for information relevant to a user question. "
            "Use when the user asks about product features, pricing, or policies. "
            "Do NOT use for general world knowledge — only internal docs."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "query": {"type": "string", "description": "Concise search query, max 15 words"}
            },
            "required": ["query"]
        }
    }
]

Key rules:

• State what the tool does AND when NOT to use it
• Constrain input fields with descriptions and formats
• One tool per responsibility — avoid Swiss-army tool descriptions

5. Classification Prompt Template (SME Rubric)

[SYSTEM]
You are a strict evaluator using the rubric below. Return only valid JSON.
Do not infer intent beyond what is stated in the rubric.

RUBRIC (locked — do not modify):
{{rubric_text}}

[USER]
Evaluate the following response.

Response: """{{response_text}}"""
Reference answer: """{{reference_text}}"""

Output format:
{
  "scores": {"criterion_1": <int 1-5>, "criterion_2": <int 1-5>, ...},
  "aggregate": <float>,
  "reasoning": "<one sentence per criterion>"
}

Failure modes to avoid:

• Leaking the rubric-modification instruction into the user turn
• Asking for scores and prose in the same turn without a separator
• Omitting the reference answer when the rubric is comparative

6. RAG Prompt Template

[SYSTEM]
Answer the user's question using ONLY the provided context.
If the context does not contain sufficient information, say "I don't have enough
information to answer that." Do not fabricate facts.

[CONTEXT]
{{retrieved_chunks}}

[USER QUESTION]
{{question}}

Checklist before shipping a RAG prompt:

• [ ] Grounding instruction is in the SYSTEM turn, not user turn
• [ ] Model is told what to say when context is insufficient
• [ ] Retrieved chunks are clearly delimited (e.g., --- between chunks)
• [ ] Output format is specified if structured output is needed

7. Common Failure Modes

8. Prompt Versioning Discipline

Treat prompts as code. A prompt change without an eval is a blind deploy.

prompts/
  classifier/
    v1.0.0.md       # baseline
    v1.1.0.md       # added CoT
    v1.2.0.md       # tightened output format
  rag/
    v1.0.0.md

Checklist before shipping a prompt change:

• [ ] New version file committed (never overwrite in place)
• [ ] Eval run on held-out set (same set across versions for comparability)
• [ ] Delta metrics recorded: accuracy, format compliance, latency, cost
• [ ] Rollback path tested (old version still in repo and deployable)
• [ ] Change summarised in commit message: what changed and why

For Claude models, also record: model version pinned, temperature, max_tokens, and any top_p / top_k settings — these affect output distribution as much as the prompt text.

References

• https://github.com/dair-ai/Prompt-Engineering-Guide
• https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/overview
• https://docs.anthropic.com/en/docs/build-with-claude/tool-use
• https://www.promptingguide.ai/