Chapter 4: Prompt Engineering for Production

1. Chapter Overview

Production prompt engineering specifies the instruction interface between an AI system and a foundation model. The work is not a search for magical phrasing; it makes requested model behavior explicit, reviewable, testable, and safe to change.

A prompt occupies one layer in Chapter 2's AI System Stack Reference Model. The surrounding system gives it a bounded task, authorized inputs, and a context package. The prompt requests model behavior and defines an output contract that a person or the harness can inspect, but it cannot create authority, guarantee truth, maintain durable state, or enforce consequential policy.

The primary engineering decision in this chapter is:

What production prompt specification will express the intended behavior without hiding responsibilities that belong elsewhere in the system?

This chapter covers prompt structure, instruction hierarchy, constraints, examples, output contracts, non-success behavior, prompt versioning, and prompt-specific checks. Context assembly belongs to Chapter 5; validation, retries, routing, state, and control to Chapter 6; evaluation strategy and launch evidence to Chapter 8; security and governance policy to Chapter 9; and release and rollback mechanics to Chapter 10.

The main artifact is the Production Prompt Specification: the reviewable source for a prompt, not merely the rendered text sent to a model.

2. Primary Engineering Problem

Prompts are easy to edit and hard to govern. A small text change can alter product behavior without touching an API, schema, or conventional code path. The same flexibility that helps during exploration creates risk in production.

As systems grow, prompt text often becomes a dumping ground for unresolved design decisions. A single prompt may be expected to classify requests, recover missing data, apply policy, choose tools, format output, remember state, and compensate for weak validation. The resulting text may work for familiar demonstrations while remaining impossible to reason about under conflict or change.

Three properties make the problem difficult.

First, instructions arrive from multiple sources. Application requirements, task definitions, run-specific parameters, user preferences, supplied documents, and examples can all use imperative language. If the interface does not separate instruction from data and define precedence, the model resolves contradictions probabilistically.

Second, natural language can conceal a contract change. Replacing “may” with “must,” adding a persuasive example, or changing an abstention rule can matter as much as changing an endpoint, yet reviewers may treat it as copy.

Third, a prompt can request behavior but cannot guarantee it. Deterministic responsibilities still need deterministic controls: the harness must parse and validate a requested structured response, and the system must enforce an access boundary before it assembles context or executes an action.

The engineering objective is therefore not the shortest or most eloquent prompt. It is a prompt specification whose sources, rules, conflicts, outputs, and change impact a team can review before production behavior changes.

3. Core Mental Model

A production prompt is an instruction contract with five parts:

1. Purpose: the single bounded job the interaction is meant to perform.
2. Instruction plane: the ordered requirements and preferences that govern the job.
3. Data plane: user content, evidence, and other inputs to operate on.
4. Outcome contract: the allowed forms of success, clarification, abstention, refusal, or handoff.
5. Traceability: the owners, requirement identifiers, version, assumptions, and prompt-specific review cases that explain why the contract exists.

The instruction plane and data plane need a visible boundary. A document that says “ignore previous instructions” may be evidence to summarize, not an instruction to follow. Likewise, a user's submission may contain both a legitimate goal and material to transform. The prompt specification defines how the system labels and assembles those regions. Chapter 9 examines malicious manipulation; this chapter establishes the construction discipline required even when every participant is well intentioned.

A vendor-neutral precedence model

Model APIs expose different role names and message formats, so a production design should describe precedence semantically rather than depend on one vendor's labels. A useful model, from highest to lowest authority, is:

1. Application invariants: requirements that apply to every invocation of this prompt, such as its non-goals, allowed task class, and mandatory outcome contract.
2. Task contract: instructions for the bounded job, including decision criteria, constraints, and output requirements.
3. Run parameters: authorized configuration for this invocation, such as audience, locale, length limit, or operating mode.
4. User goal and preferences: what the user wants within the permitted task and which optional qualities they prefer.
5. Supplied evidence: material the model may analyze or transform but must not treat as governing instruction.
6. Examples: demonstrations of intended behavior that clarify requirements but do not override them.

This order is not universal. A team may choose another sequence, provided it records the choice and gives every instruction source an owner, scope, priority, and conflict rule. Text order alone is not a conflict model.

When two requirements at the same level conflict, the model cannot be expected to infer organizational intent. The specification must identify which rule wins, narrow their scopes so they no longer overlap, or define an outcome such as clarification or handoff. A conflict that survives design review is an unresolved product decision, not a prompt-tuning problem.

Decision-ready prompt design

A prompt specification is decision-ready when reviewers can answer:

• What single behavior does this prompt own, and what are its explicit non-goals?
• Which requirement justifies each major instruction?
• Which sources may issue instructions, and which are only data?
• How are hard requirements distinguished from preferences?
• What happens when inputs are missing, malformed, or contradictory?
• What output contract does the downstream consumer require?
• Which non-success outcome applies under each known condition?
• What assumptions about context, model capability, and harness behavior remain external?
• Which prompt-local checks demonstrate that the specification is internally coherent?

4. System Boundary

The prompt layer shapes one model interaction through instructions. It may specify the task, interpretation rules, constraints, decision criteria, response form, examples, and conditions for non-success outcomes.

It does not own retrieval, ranking, compression, durable memory, orchestration, tool execution, authorization, deterministic validation, full evaluation strategy, release topology, or broad governance.

The following boundaries keep a prompt from substituting for system design.

A prompt can request; the system must enforce

State expected behavior in the prompt, but enforce invariants with deterministic controls. Validate malformed output before it reaches a consumer. Exclude records the user may not access before the interaction begins. Enforce required approval in the action path.

The prompt specifies context assumptions; it does not assemble context

The specification may say that the interaction requires current policy excerpts with provenance and an authorization decision. Chapter 5 determines how those inputs are selected, authorized, ranked, and packaged.

The prompt defines outcome semantics; the harness controls the next state

The prompt may define a structured abstention or clarification response. Chapter 6 determines whether the system accepts, retries, escalates, or converts that response into a user interaction.

Prompt checks are not a ship decision

Prompt-specific checks can show that required sections render, conflicts produce intended outcomes, and named response cases behave plausibly. Chapter 8 determines representative coverage, scoring, regression methodology, thresholds, and whether the complete system is good enough to ship.

Prompt identity is not a release

This chapter defines what makes a prompt asset a distinct version and what dependencies a change may affect. Chapter 10 owns packaging prompt versions with model, context-index, harness, and configuration versions; progressive exposure; rollback; and CI/CD.

5. Design Principles

Principle 1: Give every prompt one primary job

Describe the job as a decision or transformation for a named consumer. “Extract contract dates for the renewal workflow” is reviewable; “read the document and help the business” is not.

Record non-goals alongside the goal. If the prompt extracts dates but does not determine legal enforceability, say so. This boundary prevents helpful-sounding behavior from silently expanding the product's decision surface.

Principle 2: Make instruction precedence structural

Separate application invariants, task instructions, run parameters, user input, evidence, and examples in both the specification and the rendered interaction. Give each instruction source an owner and scope, then define what happens when sources disagree.

Do not rely on proximity, capitalization, or emphatic wording to establish authority. “IMPORTANT” is typography; precedence is architecture.

Principle 3: Classify constraints by semantics

Not every instruction is equally binding. Use explicit categories:

• Requirements: conditions that must hold for an acceptable response.
• Prohibitions: behavior or content that must not appear.
• Decision criteria: rules used to choose among valid outcomes.
• Preferences: qualities to optimize when they do not conflict with requirements.
• Presentation guidance: tone, length, and formatting for human use.

Classification turns a vague instruction conflict into a design question. If brevity is a preference and evidence is required, evidence wins. If two requirements conflict, correct the specification or define a non-success outcome.

Principle 4: Design the output for its consumer

Begin with the response consumer and the decision that follows. A human-facing explanation, a machine-routed classification, and an audit record require different contracts.

For structured output, define required, optional, nullable, and prohibited fields; allowed values; whether extra fields are permitted; and how uncertainty or insufficient evidence is represented. Give the contract an identity or version when another component depends on it. Preserve human-readable fields where a person must understand the basis of a result.

The prompt requests this contract; the harness remains responsible for parsing, validation, repair, and rejection.

Principle 5: Distinguish clarification, abstention, refusal, and handoff

These outcomes are not synonyms:

For each applicable outcome, the prompt specification defines the trigger and requested response shape. The harness owns the transition that follows, while Chapter 9 owns the source and governance of policy rules.

Principle 6: Use examples as requirement evidence

Select examples for coverage, not volume:

• a canonical positive example shows ordinary success
• a boundary example shows behavior near an important decision threshold
• a clarification, abstention, or refusal example demonstrates a non-success outcome
• a counterexample shows a plausible but unacceptable response and names the violated requirement

Trace every example to a requirement or named edge case, and remove irrelevant details the model could imitate as accidental rules. If an example contradicts the prose, fix the specification; do not expect reviewers or models to infer which one is authoritative.

Principle 7: Version the specification as a behavioral asset

A new prompt version is warranted when a change alters instructions, precedence, variables, constraints, examples, outcome semantics, or the output contract. Record the owner, rationale, affected requirement identifiers, dependency assumptions, and compatibility implications.

Classify changes by impact, not text size. Fixing a typo in a comment may be editorial. Changing an allowed value, adding a counterexample that shifts classification, or turning abstention into a best-effort answer is behavioral. Even a one-word edit may require a new version and broader review.

Principle 8: Keep prompt-specific checks local and traceable

Chapter 4 owns checks of the prompt artifact and bounded interaction:

• required-section and undeclared-variable checks
• rendering with complete, missing, empty, and malformed inputs
• review cases for instruction conflicts
• examples of every allowed outcome shape
• smoke interactions mapped to named prompt requirements
• compatibility checks for a changed output contract

These checks expose specification and assembly defects. They feed the system evaluation plan but do not establish representative quality or launch readiness.

6. Architecture Patterns

These are prompt-interface patterns, not orchestration patterns.

Pattern 1: Single-purpose instruction contract

One specification owns one bounded model task and one outcome contract. This default keeps responsibility, change impact, and review cases legible. When decisions require different evidence or acceptance rules, separate them at the system-design level instead of hiding a workflow in one prompt.

Pattern 2: Layered instruction interface

Application invariants, task instructions, run parameters, user content, evidence, and examples occupy visibly distinct regions with documented precedence. Use this pattern when several legitimate parties influence an interaction, accepting the cost of explicit assembly and conflict analysis.

Pattern 3: Schema-first response contract

The downstream consumer's needs define the output before anyone writes prompt wording. Outcome type, field semantics, uncertainty representation, and compatibility rules become interface decisions. The pattern suits machine consumption, but accountability may still require a human-readable rationale.

Pattern 4: Criteria-and-boundaries block

For an ambiguous judgment, the specification groups requirements, prohibitions, decision criteria, preferences, and boundary cases. This reduces the chance that tone guidance or examples silently override the actual decision rule.

Pattern 5: Requirement-linked example set

Maintain a small example set tied to requirements and edge cases. Reviewers should be able to see what each example teaches and remove those that add noise. Include non-success behavior whenever clarification, abstention, refusal, or handoff forms part of the contract.

Pattern 6: Policy-referenced instruction interface

The prompt explains how to use an identified policy source supplied as evidence or configuration without embedding an unowned copy. It records the expected policy identity and the response to a missing or incompatible source. Chapter 5 owns context assembly, Chapter 9 policy governance, and Chapter 10 runtime dependency control.

Multiple prompt contracts may be composed into a larger workflow, but sequencing, state, acceptance, and retry logic belong to the harness rather than to this pattern catalog.

7. Failure Modes

Failure mode 1: Prompt as dumping ground

Retrieval assumptions, workflow state, policy enforcement, validation, and tool control accumulate in the prompt. Reviewers can no longer identify its primary job or tell which instructions merely compensate for missing system components.

Failure mode 2: Precedence by accident

Text order, emphasis, examples, and user phrasing imply different priorities. The model resolves the conflicts inconsistently, while the team blames model randomness instead of the undefined interface.

Failure mode 3: Evidence promoted to instruction

Supplied content is placed in the instruction plane or left undelimited. Imperative text inside a document then changes behavior even though the document was meant only to be analyzed.

Failure mode 4: Output contract by convention

Downstream code expects fields, values, or prose patterns the specification never defines. A small wording change then causes a parse failure or, worse, a valid-looking semantic error.

Failure mode 5: Best-effort answer as the only outcome

Clarification, abstention, refusal, and handoff are collapsed into generic apology text or omitted entirely. The model invents answers when evidence is missing, asks unnecessary questions when it should abstain, or refuses tasks that merely require authorization elsewhere.

Failure mode 6: Example overfitting

Examples contain incidental names, numbers, ordering, or style that the model imitates. Coverage looks strong for near-duplicates while behavior at the actual boundary remains unspecified.

Failure mode 7: Unversioned behavioral change

An edit ships as copy even though it changes precedence, output compatibility, refusal semantics, or decision criteria. Reviewers can no longer reconstruct which behavior was intended for a given system result.

Failure mode 8: Prompt checks mistaken for evaluation

After the artifact renders and passes a handful of smoke cases, the team treats it as production-ready. Representative scenarios, scoring, adverse cases, system interactions, and launch thresholds remain unexamined.

8. Tradeoffs

Specificity versus adaptability

Specific requirements reduce ambiguity and make review easier, but excessive prescription can block legitimate variation and make the prompt brittle. Allow adaptability where several outputs are valid; impose hard constraints where the consumer or consequence requires them. Do not turn every preference into a requirement.

Natural language versus structured output

Natural language preserves nuance and is easier for people to consume, while structured output supports deterministic validation and routing. A hybrid contract often works best: stable outcome and evidence fields around a human-readable explanation. Let the consumer determine the choice rather than defaulting to a schema.

Embedded rules versus referenced sources

Embedding rules makes the interaction self-contained, but the duplicated policy can drift. Referencing an identified source reduces duplication at the cost of a context dependency and defined missing-source behavior. Either way, high-consequence rules require enforcement outside the prompt.

Examples versus rules

Rules are compact and reviewable; examples reveal boundaries that prose may miss. Too many examples consume attention and introduce accidental patterns, so use rules for the contract and examples for its highest-value boundaries.

One interface versus composed interfaces

A single prompt reduces assembly and operational overhead. Separate contracts isolate responsibilities and outputs, but their composition becomes a harness concern with added latency, state, and failure paths. Split the interface only when independently reviewable decisions justify that cost.

Iteration speed versus change governance

Fast prompt edits accelerate learning. Once a prompt affects users, downstream systems, risk, or contractual behavior, its change impact matters more than editing speed. Apply lightweight review to editorial changes and stronger review to changes in requirements, outcomes, examples, or compatibility.

9. Production Checklist

Before approving a production prompt specification, confirm:

• The prompt has one primary purpose, an identified consumer, and explicit non-goals.
• Major instructions trace to requirement or decision identifiers.
• Application invariants, task instructions, run parameters, user content, evidence, and examples are distinguishable.
• Every instruction source has an owner, scope, priority, and conflict rule.
• Requirements, prohibitions, decision criteria, preferences, and presentation guidance are classified.
• Input variables have types, defaults where appropriate, and missing-input behavior.
• Required context assumptions are stated without embedding retrieval design.
• The output contract defines success and applicable non-success outcomes.
• Required, optional, nullable, prohibited, and extra-field semantics are explicit.
• Uncertainty and incomplete evidence have defined representations.
• Clarification, abstention, refusal, and handoff have distinct triggers and response shapes.
• Examples map to requirements or named edge cases and contain no accidental rules.
• The prompt is not the sole enforcement mechanism for authorization or consequential policy.
• Version identity, owner, rationale, affected requirements, and compatibility notes are recorded.
• Prompt-specific checks cover rendering, conflicts, outcome shapes, and named requirements.
• Known model, context, output-consumer, and harness assumptions are documented.
• Full evaluation, security, release, and operational reviews are handed to their owning disciplines.

10. Design Review Questions

1. What single behavior does this prompt own, and what does it explicitly not own?
2. Who consumes the output, and what decision or action follows?
3. Which requirement justifies each major instruction or example?
4. Which sources may issue instructions, and which are only data?
5. What wins when application, task, run, and user instructions conflict?
6. Which statements are hard requirements, decision criteria, preferences, or presentation guidance?
7. What happens when a required input or context dependency is absent or contradictory?
8. How does the output contract represent uncertainty, insufficient evidence, and incompatibility?
9. When should the result be clarification, abstention, refusal, or handoff?
10. What deterministic enforcement is incorrectly being delegated to the prompt?
11. Which examples cover actual boundaries, and which introduce irrelevant patterns?
12. Is this change editorial, behavioral, or contract-breaking, and which dependencies need review?
13. Which prompt-local checks map to the changed requirements?
14. What evidence must be produced by Chapter 8's evaluation process before the system can ship?

11. Main Artifact: Production Prompt Specification

Use this specification for any prompt that affects production behavior. Its fields define the reviewable interface, even when the rendered interaction arranges them differently.

A. Identity and Scope

• Prompt name:
• Prompt specification version:
• Owner and reviewers:
• Source AI Fit and Risk Assessment Worksheet and version:
• Accepted product decision IDs, constraints, and validity limits consumed:
• Primary purpose:
• Downstream consumer:
• Supported system behavior:
• Explicit non-goals:
• Requirement or decision IDs:
• Change rationale:
• Change classification: editorial / behavioral / contract-breaking

B. Dependencies and Assumptions

• Expected model capabilities:
• Required context package and provenance assumptions:
• Expected policy or rule-source identity:
• Dependent harness behavior:
• Output consumer and supported contract versions:
• Known limitations:

These fields identify dependencies; they do not replace the Context Assembly Plan, Harness Control-Loop Diagram, Tool Permission Matrix, Evaluation Plan, AI Threat Model, or AI Release Manifest owned by Chapters 5–10.

C. Input Variables

Also record:

• user-goal region:
• supplied-evidence region:
• disallowed inputs:
• delimiter or assembly expectations:

D. Instruction Sources and Precedence

• same-priority conflict behavior:
• unresolved-conflict outcome:
• instruction/data separation rule:

E. Constraints and Decision Rules

• Requirements:
• Prohibitions:
• Decision criteria:
• Preferences:
• Presentation guidance:
• Criteria for incomplete or ambiguous cases:

F. Outcome Contract

Success

• Output type and contract version:
• Required fields:
• Optional fields:
• Nullable fields and meanings:
• Prohibited fields:
• Allowed values:
• Extra-field policy:
• Evidence or citation fields:
• Uncertainty representation:
• Human-readable rationale requirements:
• Compatibility notes:

Non-success outcomes

Only include outcomes applicable to this prompt. The harness design defines how the system executes the downstream handling.

G. Examples

H. Prompt-Specific Checks

• known uncovered prompt cases:
• Evidence handed to the Evaluation Plan:
• review required for future changes:

I. Downstream Handoffs and Change Boundary

• Context Assembly Plan handoff: required input regions, evidence semantics, provenance assumptions, and missing or conflicting evidence behavior:
• Harness Control-Loop Diagram handoff: outcome contract, validation boundary, clarification, abstention, refusal, and handoff states:
• Evaluation Plan handoff: requirement IDs, prompt-local invariants, covered examples, known gaps, and prompt version:
• AI Release Manifest handoff: prompt asset identity, compatibility classification, dependent contract versions, and material-change class:

The Production Prompt Specification consumes the accepted system role, decision surface, outcome boundaries, and requirement IDs from the exact AI Fit and Risk Assessment Worksheet decision. Downstream consumers may demand stronger evidence, reject an outcome, or block release. They may not silently broaden the prompt's purpose, supported behavior, population, instruction authority, output guarantees, or allowed non-success behavior. Any such broadening returns to the owning product decision; the team must accept a new decision before this specification or a downstream artifact relies on it.

12. Key Takeaways

• A production prompt is a versioned instruction contract, not a container for unresolved system design.
• Instruction authority must be structural: identify sources, owners, scopes, precedence, and conflict rules.
• User content and supplied evidence belong in the data plane unless the specification explicitly grants instructional authority.
• Requirements, prohibitions, criteria, preferences, and presentation guidance should not be treated as interchangeable prose.
• Clarification, abstention, refusal, and handoff are distinct product outcomes with distinct triggers and downstream expectations.
• Output contracts should express uncertainty and compatibility, while deterministic validation remains outside the prompt.
• Prompt-specific checks establish artifact coherence; Evaluation Engineering determines whether the complete system is good enough to ship.
• The Production Prompt Specification is the source a team reviews, versions, and traces back to the accepted AI Fit and Risk Assessment Worksheet and engineering requirements.