Chapter 11: Observability, Incidents, and AI SRE

1. Chapter Overview

Launching an AI system replaces release uncertainty with operating uncertainty. Real users introduce inputs, combinations, timing, incentives, and downstream effects that no preproduction test program can exhaust. Models and dependencies change. Context sources age. Long-running workflows cross release boundaries. Costs accumulate through retries, fan-out, fallbacks, and review. Outcomes may arrive long after a response was served.

The primary engineering decision in this chapter is to define how the team will know whether the production system remains acceptably useful, reliable, economical, and controllable—and what it will do when the evidence says otherwise.

Logs alone cannot support that decision. An operable AI system needs:

• explicit operational objectives tied to user and business outcomes;
• a versioned telemetry contract and reconstructable causal execution graph;
• behavioral, execution, economic, drift, and evidence-health signals with known semantics;
• detection rules connected to owners and authorized responses;
• an incident command model that contains harm before certainty is complete;
• recovery criteria that cover behavior, state, actions, users, and delayed outcomes; and
• a learning loop that produces verified risk reduction.

This chapter connects those elements into an operating discipline. It consumes evaluation-validity decisions from Chapter 8, trust policy and security obligations from Chapter 9, release transitions from Chapter 10, and action semantics from Chapter 7. Operations makes the resulting runtime evidence decision-ready, coordinates the response, and verifies recovery without redefining those upstream contracts.

2. Primary Engineering Problem

Traditional service indicators remain necessary. Availability, latency, error rate, saturation, dependency health, queue age, and capacity still matter. They are not sufficient.

An AI endpoint can return quickly and successfully while a material customer cohort receives unsupported answers. A workflow can complete while repeating an external action. A lower-cost route can increase correction work enough to raise total cost. A quality signal can appear stable because the label feed stopped. A rollback can restore code while queued work, contaminated state, or prior user-visible effects remain.

The engineering problem is therefore:

Operate the complete production behavior using evidence whose identity, scope, delay, and limitations are understood.

Four properties make the work difficult.

Health is multidimensional. Execution, behavior, economics, and evidence can disagree. A single aggregate score hides which promise is failing and which response is safe.

The subject is composite and causal. A result may depend on a release, runtime configuration, route, prompt bundle, model resolution, context snapshot, harness path, tool policy, dependency state, workflow history, and human decision. Mutable names and request-level logs cannot reconstruct that chain.

Important evidence is incomplete or delayed. Deterministic invariants can be immediate; expert judgments and downstream outcomes may take hours or weeks. Sampling, missing labels, failed joins, and population shift all change what a signal can support.

Response changes the system under investigation. Operators may constrain exposure, pause workflows, change routing, invoke a rollback, or compensate an action. Each intervention needs authorization, a durable record, a reversal path where possible, and verification across new and already-active work.

AI SRE is the discipline of managing these properties as an operating system: translating approved expectations into operational objectives, collecting trustworthy evidence, detecting material deviations, coordinating consequence-aware response, verifying recovery, and improving the controls that made the failure reachable.

3. Core Mental Model

Outcomes supported by four operating layers

Operational design begins with the user or business outcome the system must protect, not with the metrics that happen to be easy to graph. Four operating layers support each outcome:

1. Behavioral health: whether runtime outputs and actions remain within approved expectations. Examples include deterministic contract adherence, sampled judgment results, correction rates, escalation patterns, and delayed outcomes.
2. Execution health: whether requests and workflows execute reliably. Examples include availability, latency, queue age, dependency failure, validation failure, completion, cancellation, and saturation.
3. Economic health: whether useful work is produced within an acceptable resource envelope. Examples include cost per completed eligible workflow, retry amplification, fallback share, review burden, and budget burn.
4. Evidence health: whether the data used to judge the other layers is complete, fresh, correctly joined, interpretable, and available. Examples include schema conformance, sample coverage, label delay, join success, telemetry loss, and evaluator availability.

Collapsing these layers into one health score destroys their meaning. Low latency cannot compensate for a prohibited event, and missing evidence cannot count as healthy evidence. A cost overrun does not prove poor behavior; neither does a drift signal prove degradation. Each condition retains its own semantics and response path.

The operational objective contract

An operational objective turns an expectation into a decision rule. Every objective should declare:

• the outcome or prohibited condition it protects;
• the subject and unit of analysis: response, action, session, workflow, user outcome, or downstream effect;
• the eligible population and denominator;
• required slices such as workflow, risk class, tenant, cohort, route, region, or release, where policy permits;
• the indicator, unit, aggregation, direction, and known limitations;
• the observation window and, where appropriate, allowed failure or budget;
• evidence delay, minimum volume, and completeness requirements;
• behavior when data is missing, late, or indeterminate;
• the accountable owner and operating cadence; and
• the response: observe, investigate, page, constrain, hold, or invoke another chapter's control.

The denominator is part of the objective, not an implementation detail. “Two hundred corrections” cannot be interpreted without knowing eligible interactions, sampling rules, slice distribution, and reporting delay. The same applies to error, cost, refusal, escalation, and outcome rates.

Chapter 8 establishes whether a behavioral indicator and its thresholds are valid evidence for an approved claim. Operations then establishes whether that indicator is available and correctly computed in production, and which decision follows when it changes.

Evidence-to-decision states

An objective should have at least four states:

• Healthy: sufficient current evidence supports the operating boundary.
• Degraded: sufficient evidence shows a boundary or budget is being consumed abnormally.
• Violated: sufficient evidence shows the boundary has been crossed or a prohibited event occurred.
• Indeterminate: evidence is too incomplete, late, invalid, or unavailable to establish the other states.

Indeterminate is an operational state with an owner and response. For a low-consequence feature, it may open an investigation and increase sampling. For an action-capable, high-consequence workflow, loss of the evidence required to prove safe operation may require manual supervision or paused admission. The choice belongs in the runbook before an incident.

Causality before correlation

Dashboards reveal patterns; causal identity makes diagnosis possible. Operators need to follow a unit of work from entry through routing, context assembly, model invocation, validation, queues, approvals, external actions, state transitions, user-visible completion, and delayed outcome. That path also has to retain the effective system identity and relevant changes over time.

The purpose is not to store every payload. It is to preserve enough identity, structure, protected evidence, and authoritative receipts to answer:

1. What happened?
2. To which population and workflows?
3. Under which effective system state?
4. What evidence is complete, sampled, delayed, or missing?
5. Which action can reduce harm safely?
6. How will recovery be proven?

4. System Boundary

Observability records; evaluation judges

Operations runs measurements that Chapter 8 has defined or accepted. The operational record carries population, sampling, evaluator identity, rubric or metric reference, delay, uncertainty reference, and validity limitations. This chapter neither designs rubrics nor constructs golden datasets, and it never treats a proxy as proof of correctness.

When production drift or an incident calls prior evidence into question, operators preserve the affected population and effective-state identities, then request reevaluation. Chapter 8 decides whether the evidence remains valid.

Operations coordinates releases; deployment executes transitions

Operations may conclude that exposure should be held, traffic reassigned, configuration overridden, or a release rolled back or forward. Chapter 10 defines and executes those transitions and emits effective-state receipts. The incident record explains why the change was requested, observes its effect, and verifies recovery without recreating deployment mechanics in the runbook.

Operations observes actions; tool design defines their semantics

The causal graph records action intent, authorization, idempotency identity, execution receipt, observed effect, cancellation, and compensation references using Chapter 7's semantics. During containment, operations coordinates queued and in-flight actions and verifies downstream effects; the incident is too late to invent compensation behavior.

Trust policy governs operational evidence

Operational data can include prompts, retrieved content, user identifiers, tool arguments, outputs, reviewer notes, error bodies, and derived attributes. Chapter 9 determines permitted collection, access, retention, residency, deletion, export, emergency access, and security response. Operations enforces those decisions across telemetry, incident collaboration, forensic preservation, and replay.

Unless a defined purpose and scope have been approved, payload capture is denied. Searchable structural metadata stays separate from protected payload stores. Copying traces into tickets, chat, documents, or screenshots creates additional stores subject to the same policy.

Operations does not become a second source of truth

Telemetry references authoritative release, policy, consent, action, and business records; it does not replace them. Because logs may be delayed, sampled, transformed, or dropped, every operational query identifies which fields are authoritative receipts and which are observational copies.

Incidents include behavioral and economic harm

An incident may be downtime, semantic degradation, prohibited content, repeated action, context leakage, cost amplification, persistent bias in a material slice, loss of required evidence, or an inability to control an active workflow. Security-specific command and notification obligations come from Chapter 9, but the operational incident system must recognize and route the handoff.

5. Design Principles

Principle 1: Define health before choosing dashboards

Instrument promises and prohibited outcomes, not tool defaults. Every dashboard and alert traces back to an operational objective, a dependency needed to interpret it, or an evidence-health condition. A metric with no owner or associated decision is diagnostic data, not an operating control.

Principle 2: Version telemetry as a production contract

Telemetry producers and consumers depend on stable meanings. Version events and fields, define units and timestamps, state null and retry semantics, record sampling and sensitivity, and test producer conformance alongside critical queries. A silent instrumentation change can be as operationally dangerous as an application regression.

Principle 3: Preserve causal identity across time and boundaries

A request is only one unit of work. Stable identities have to survive sessions, durable workflows, queues, retries, fan-out, fan-in, human approvals, actions, state changes, streaming, cancellation, and delayed outcomes. Each step records effective system identity rather than relying on mutable labels.

Principle 4: Keep signal classes distinct

Deterministic invariants, heuristic proxies, sampled judgments, explicit feedback, observed user behavior, and delayed outcomes answer different questions. Their displays and alerts need to expose evidence strength, delay, sampling, and blind spots. No aggregate should average away a prohibited event or present a proxy as direct quality evidence.

Principle 5: Treat cost and drift as operational signals with semantics

Attribute cost to an eligible unit of useful work and expose retries, fan-out, loops, fallback, review, and abandoned work. A drift signal names the changed subject, reference population, expected variation, affected slices, and triage path. Token counts and distribution distance are measurements, not operational conclusions.

Principle 6: Make evidence failure visible

The observability system needs its own monitoring. Missing denominators, late outcomes, broken joins, dropped events, stale dashboards, unavailable evaluators, and failed redaction or deletion jobs all produce explicit evidence-health states. Critical controls also need a declared degraded mode and, where consequence requires it, an independent signal path.

Principle 7: Alert only when a decision is expected

A page represents an urgent condition for which the recipient can make a useful decision now. Lower-confidence or delayed signals usually belong in an investigation or review queue. The rule identifies its protected objective, evidence requirements, severity, route, deduplication, allowed automation, test, and review history.

Principle 8: Contain consequence before completing diagnosis

When harm may be ongoing, waiting for root-cause certainty can cost more than a bounded reversible intervention. The incident lead selects the smallest control likely to reduce consequence, preserves evidence, sets an expiry, and defines verification. Broader actions demand broader authority and stronger evidence.

Principle 9: Recover behavior, state, and effects—not only infrastructure

Recovery covers new admission, queued and in-flight work, persisted state, caches, external effects, user remediation, telemetry health, and delayed outcomes. Normal error rate and latency prove only that part of the system has recovered.

Principle 10: Close incidents with verified risk reduction

Every follow-up identifies the reachable condition or missing control it changes, its owner, the verification method, and the authority that may accept residual risk. A closed ticket says nothing about whether the system improved.

6. Architecture Patterns

The operating architecture uses cooperating planes that may have separate systems and owners but share identity, semantics, and response contracts.

Pattern 1: Instrumentation plane and observability contract

The instrumentation plane emits versioned events and authoritative references from application, model, context, harness, workflow, action, and dependency boundaries. Its core artifact is an observability contract containing:

• schema and event identity, producer, owner, purpose, version, and compatibility policy;
• request, session, conversation, workflow, step, attempt, span, action, outcome, and incident identifiers;
• parent-child, follows-from, spawned-by, resumed-from, retry, fan-out, fan-in, and compensation links;
• event time, ingest time, sequence information, clock source, and expected lateness;
• immutable release, effective configuration, route decision, prompt bundle, model resolution, context snapshot, harness, tool-policy, topology-cell, dependency, tenant, cohort, and risk-class identities where permitted;
• fields, units, aggregation and retry rules, required and optional status, and null, unknown, and not-applicable semantics;
• sensitivity, transformation or redaction state, permitted use, retention class, and protected payload reference;
• sampling rule, inclusion probability or weight, exclusions, and whether the record may support aggregate claims; and
• validation, rollout, producer-consumer compatibility, and deprecation tests.

Where practical, the contract becomes executable through producer schema checks, representative trace tests, join tests, query fixtures, and alerts for incompatible change. High-cardinality identity is retained only when its diagnostic value and policy permit it, yet aggregation must not make high-consequence events impossible to reconstruct.

Pattern 2: Causal execution graph

A trace for synchronous inference is a tree. An operational record for a production AI workflow is usually a graph.

The graph represents entry requests, sessions, conversations, workflows, steps, attempts, queues, model calls, context assemblies, validation, approvals, actions, state transitions, streaming, cancellations, terminal outcomes, and delayed outcome joins. Queue events include enqueue, dequeue, age, lease, timeout, deduplication, and dead-letter transitions. Human approval records bind the decision, authority, expiry, and exact proposed action. Action spans reference Chapter 7 receipts from intent through observed effect and any compensation.

Every node records the effective composite identity that applied at that point. Without it, operators cannot explain a dynamic dependency resolving differently, a long-running workflow retaining release affinity, or a runtime-state change made during an incident.

Even when payloads cannot be retained, the graph can preserve stable hashes, structural properties, source and policy decisions, action receipts, state-version expectations, protected evidence references, and known reconstruction limits. “Reproducible” must say whether the team can perform exact replay, behavioral replay with protected or synthetic inputs, or only structural reconstruction.

Pattern 3: Telemetry and evidence-health plane

This plane collects, transforms, redacts, samples, stores, joins, retains, and serves telemetry. It is a production dependency with its own objectives:

• producer schema conformance;
• event loss, duplicate rate, ordering, and clock skew;
• end-to-end freshness and backlog;
• sampling coverage, especially for prohibited events, high-risk paths, and action receipts;
• collector, buffer, transformation, storage, index, and query saturation;
• dimensional cardinality and aggregation loss;
• join success among traces, releases, costs, feedback, reviews, and delayed outcomes;
• dashboard and alert rule version, test, deployment, and rollback status; and
• access, redaction, retention, deletion, export, and legal-hold job success.

“No data” must never render as zero. If a quality feed becomes late, the behavioral objective becomes indeterminate according to its contract. If the primary telemetry path is unavailable during a severe incident, the runbook identifies independent infrastructure, action, or business receipts and the system's fail-open, fail-closed, hold, or manual-supervision behavior.

Pattern 4: Operational-state plane

The operational-state plane correlates effective system identity, execution, behavior, economics, feedback, outcomes, and changes. It provides role-oriented views rather than one universal dashboard:

• operators see active objectives, alerts, evidence health, scope, dependencies, and available controls;
• domain owners see behavioral slices, sample and outcome delay, feedback, drift, and affected composites;
• release owners see effective-state and traffic changes from Chapter 10;
• action owners see queued, in-flight, completed, duplicated, and compensated work from Chapter 7; and
• incident leads see an authoritative timeline, hypotheses, decisions, actions, and recovery evidence.

Each view exposes its query version, denominator, freshness, sampling, and known blind spots. Release, route, configuration, model resolution, source, policy, traffic, price schedule, evaluator, and telemetry-schema changes appear on the same time axis, where operators can test correlation without mistaking it for cause.

Pattern 5: Behavioral, economic, and drift signal contracts

Behavioral signal contract

Every behavioral signal carries the product claim or failure condition it informs, along with its signal class, subject, eligible population, denominator, slices, sampling frame, inclusion probability, delay, revision behavior, join key, Chapter 8 validity reference, baseline, expected direction, freshness, completeness, blind spots, owner, and allowed operational use.

Signal classes include:

• deterministic invariant or contract failure;
• heuristic proxy;
• sampled automated or human judgment;
• explicit user feedback;
• observed user behavior; and
• delayed business or downstream outcome.

An immediate validation failure may page when it represents a prohibited action. A sampled quality judgment opens an investigation only after sufficient volume arrives. A delayed outcome may verify recovery even though it arrives too late for immediate containment. The contract keeps these signals from being treated as interchangeable.

Economic signal contract

Economic health starts with an eligible unit of value or useful work: a completed research workflow, reviewed case, resolved request, or another product-owned unit. The cost record attributes model and context processing, retrieval, tools, evaluators, storage, network, queues, shared serving, and expert review where material. Decision-time estimates are later reconciled against authoritative billing under the applicable price-schedule identity.

Segment cost by tenant, feature, workflow, cohort, release, route, model, retry path, and outcome where permitted. Track input, output, cached work, duration, concurrency, retry, fan-out, fallback, and loop amplification. Distinguish completed, abandoned, failed, duplicated, and compensated work. Budgets should define burn rate, forecast, anomaly conditions, owner, and authorized containment when cost data is missing or delayed.

Drift signal contract

A drift signal names its subject and change class: input population, context corpus, dependency, model behavior, route mix, workflow, outcome, policy, label, or telemetry. Its contract identifies the reference population and window, validity assumptions, refresh authority, representation, unit, slices, cadence, seasonality, expected change, minimum volume, uncertainty, repeated-alert handling, freshness, late-data behavior, correlations, owner, and action.

A detected difference is classified before action: expected change, data defect, instrumentation defect, applicability warning, quality-regression candidate, or incident. Chapter 11 detects and routes the condition. Chapter 8 decides whether the shift invalidates evidence or requires a new evaluation.

Pattern 6: Detection and decision plane

Each detection rule is a small operational decision contract:

• protected objective or prohibited event;
• subject, population, denominator, slice, baseline, threshold, direction, and window;
• evidence delay, minimum volume, completeness, and indeterminate behavior;
• symptom versus suspected cause and related dependency or evidence-health signals;
• severity, destination, owning service and domain owner, acknowledgement target, and escalation path;
• grouping, deduplication, suppression, maintenance-window, and repeat-notification behavior;
• multi-window or budget-burn behavior where useful;
• allowed automated action, preconditions, authority, cooldown, expiry, reversal, and safety stop;
• runbook reference, test method, last exercise, and change history; and
• false-positive, false-negative, missed-incident, and stale-rule review.

Automated authority grows only as evidence quality, consequence, scope, and reversibility allow. A deterministic prohibited event may justify immediate scoped containment; a delayed sampled proxy should not disable a high-value service. When several rules can act, one coordinator prevents oscillation and conflicting controls.

Pattern 7: Response plane and incident command

The response plane combines incident command with controls owned across the system. Its consequence-aware lifecycle is:

1. Accept a signal, user report, or operator concern even if classification is uncertain.
2. Declare an incident with provisional class, scope, and severity.
3. Assign an incident commander, operations lead, domain lead, communications lead, and scribe as consequence requires.
4. Bound affected workflows, populations, composites, tenants, regions, time, actions, and possible harms.
5. Activate evidence-preservation and data-handling instructions.
6. Evaluate containment options and authorize a bounded action.
7. Record observations, hypotheses, decisions, commands, approvals, effects, and communications on one timeline.
8. Restore or constrain service to an explicitly accepted degraded mode.
9. Verify technical, behavioral, state, action, user, telemetry, and delayed-outcome recovery.
10. Close only when residual risk, follow-up ownership, communication, and review obligations are explicit.

Severity includes reachable consequence, authority crossed, exposure, duration, reversibility, affected population, ongoing harm, data sensitivity, economic loss, and uncertainty. A rare repeated financial action may outrank a widespread cosmetic failure. High uncertainty can increase severity when potential harm is large and observability is weak.

Containment hierarchy

Prefer the narrowest action that can materially reduce consequence:

1. observe, increase sampling, or narrow uncertainty;
2. constrain a feature, workflow, cohort, tenant, region, route, dependency, or action class;
3. require review or switch to read-only, draft-only, or another declared degraded mode;
4. stop admission, pause queues or schedulers, revoke action eligibility, or isolate state;
5. request a Chapter 10 exposure hold, reassignment, configuration override, rollback, or roll-forward; and
6. invoke Chapter 7 cancellation, deduplication, reconciliation, or compensation for external actions.

A containment action is incomplete without its scope, authority, preconditions, dependencies, expected effect, start, expiry, reversal, preserved evidence, and verification query. Emergency access and evidence handling remain governed by Chapter 9. Temporary controls either expire or move through the normal release or policy path.

Forensic evidence package

The incident record preserves:

• incident identity, scope hypotheses, authoritative timeline, sources, clock offsets, and gaps;
• alert states, acknowledgements, automated actions, operator commands, approvals, and communications;
• effective release, configuration, routing, topology, context-index, model-resolution, policy, and dependency identity over time;
• representative causal graphs, action receipts, state transitions, queue state, and affected-record references;
• versioned queries, result snapshots, denominators, sampling, and telemetry completeness;
• protected payload references, hashes, redaction state, authorized access, retention holds, and chain of custody where required;
• hypotheses, disconfirming evidence, decisions, mitigation observations, and unresolved uncertainty; and
• reproduction attempts and whether they constitute exact replay, behavioral replay, or structural reconstruction.

The goal is decision-grade reconstruction, not unlimited retention.

Pattern 8: Recovery and learning plane

Recovery is a set of claims supported by explicit queries or receipts:

• new traffic follows the intended effective state;
• queued and in-flight workflows are paused, migrated, completed, cancelled, or reconciled as intended;
• persisted state, caches, indexes, and workflow affinity are compatible;
• external actions are accounted for and duplicated or harmful effects remediated;
• behavioral indicators recover for affected populations and critical slices;
• evidence-health indicators are current and trustworthy;
• affected users or downstream parties receive required correction or support; and
• delayed outcomes remain within the observation window or residual uncertainty is explicitly accepted.

After containment and recovery, the learning review separates event and impact, detection, response, contributing conditions, failed or absent controls, feasible counterfactuals, and residual uncertainty. Actions are classified as immediate repair, systemic prevention, detection improvement, recovery improvement, evidence improvement, or accepted residual risk.

Each action has an owner, due date, priority, verification method, release or policy path, and closure authority. Evaluation cases return to Chapter 8; trust changes to Chapter 9; release changes to Chapter 10; action-control changes to Chapter 7; and prompt, context, or harness defects to their respective owners. A recurrence review tests whether the change actually improved the system.

Compact operational walkthrough

Suppose a route-mix change increases fallback frequency. Infrastructure remains green, but cost per completed workflow rises and a delayed quality signal degrades for one high-value cohort.

After confirming billing freshness and denominator stability, the economic rule opens an investigation. A cohort behavioral rule later reaches its minimum volume and completeness conditions, so it pages. The incident lead correlates both changes with the effective route receipt, verifies the label and join pipelines, and scopes the affected workflows. The team uses Chapter 10 controls to constrain that cohort to a previously approved route, then pauses queued high-consequence actions pending Chapter 7 reconciliation. The containment record captures scope, authority, expiry, and expected effect.

Changing the route mix is containment, not recovery. Operators still verify new assignments, queued work, unit cost, behavioral results for the affected cohort, telemetry completeness, and later outcomes. The review sends a new scenario to Chapter 8, routes a policy change through Chapter 10, and adds an evidence-health dependency to the alert. A future label outage will then produce an indeterminate state instead of a healthy one.

The sequence matters: validate evidence, correlate effective state, declare and scope the incident, contain with authority, account for active work, verify every recovery dimension, and improve the controls that own the failed condition.

7. Failure Modes

Failure mode 1: Green infrastructure hides behavioral degradation

A material slice receives poor or harmful results while aggregate latency and availability remain normal. Outcome protection has failed, but the aggregate masks it because a slice denominator, direct quality signal, or delayed outcome is absent. Constrain the affected slice, confirm evidence health, and verify behavior—not only request success—after the change.

Failure mode 2: Causal identity breaks across asynchronous work

Queue boundaries, retries, fan-out, approvals, or resumed workflows lose the original identity. Operators double-count attempts, misattribute cost, or cannot determine which action occurred. Preserve durable workflow and action identities, treat reconstruction as incomplete, and constrain high-consequence work until authoritative receipts reconcile it.

Failure mode 3: Biased or incomplete evidence looks precise

Risk sampling excludes confidently wrong outputs, labels arrive only for engaged users, a denominator changes, or joins silently fail. The visible metric stays stable while its covered population shrinks. Evidence-health objectives invalidate the apparent conclusion and move the signal to an indeterminate state.

Failure mode 4: Telemetry loss is rendered as health

A schema change drops events, collectors backpressure, a query times out, or an evaluator fails; the dashboard nevertheless shows zero failures. Independent pipeline-health signals, explicit no-data semantics, and critical-control degraded modes keep absence from satisfying the objective.

Failure mode 5: Cost amplification hides behind averages

Per-request cost remains flat while workflows become longer, retries and fan-out grow, work is abandoned, or billing arrives late. Measure cost per eligible completed unit, expose amplification and outcome state, reconcile estimates, and verify that containment does not create a larger operational or user cost.

Failure mode 6: Drift is misclassified

Seasonal demand or an approved source change pages as degradation, while a serious cohort shift disappears in the aggregate. Name the subject, reference, expected change, slices, volume, and uncertainty. Investigate correlation before treating difference as failure; request Chapter 8 review when applicability may have changed.

Failure mode 7: Alerts do not correspond to decisions

Noisy proxies page repeatedly, related rules create an alert storm, or suppression hides a real incident. Operators eventually ignore the system. Route lower-confidence evidence to investigation, group related symptoms, review missed incidents, and require every page to name the expected decision and owner.

Failure mode 8: Automated containment oscillates

Several rules act on stale or incomplete evidence, alternately changing routes, throttles, or admission. The system becomes less stable while its evidence becomes harder to interpret. One coordinator, bounded authority, cooldown, expiry, reversal, and a safety stop keep the controls coherent. Evidence or consequence outside that contract requires human authorization.

Failure mode 9: Containment misses active work and effects

New admission stops, but queues, schedulers, cached outputs, in-flight tool calls, state writes, or previously authorized actions continue. Scope by workflow and action state, invoke Chapter 7 reconciliation, isolate incompatible state, and verify downstream effects before restoration.

Failure mode 10: Recovery is declared too early

Error rate recovers while harmful behavior, polluted memory, incompatible state, user remediation, or delayed outcomes remain unresolved. Recovery criteria must cover each affected dimension and retain an accepted residual-risk decision for evidence that cannot yet arrive.

Failure mode 11: Incident evidence becomes unsafe or irreproducible

Operators copy sensitive payloads into uncontrolled collaboration tools, queries mutate without versioning, or retention jobs delete the only reconstruction evidence. Use protected references, query snapshots, scoped access, approved holds, and an authoritative timeline under Chapter 9 policy.

Failure mode 12: Local fixes and follow-up decay

A prompt patch fixes an exemplar, more alerts are added, and tickets close without verifying reachability or recurrence. Classify countermeasures by the control gap they close, route changes to the owning chapter, define verification, and require explicit authority to accept residual risk.

8. Tradeoffs

Operational tradeoffs should be decided using consequence, evidence delay, reversibility, population reach, workflow duration, sensitivity, signal validity, operator capacity, telemetry cost, detection speed, and recovery objectives.

Full-fidelity versus sampled tracing

Keep complete structural and identity events for critical paths, prohibited outcomes, actions, and authoritative receipts; reserve sampling for high-volume diagnostic detail. High-consequence or poorly understood workflows warrant greater fidelity. Sampling probabilities stay in the record, and rare failures remain protected so aggregate claims are interpretable. The service and data owners approve deviations from this default.

Payload visibility versus minimization

The normal posture is structured allowlisted metadata plus protected payload references, not broad raw capture. Capture expands only for an approved purpose, scope, duration, and population, with access logging and expiry. Chapter 9 policy determines that authority; incident urgency does not erase the boundary.

Immediate proxies versus delayed outcomes

Use fast proxies for detection and direct or delayed outcomes for verification, with the evidence class visible in both cases. A proxy may justify investigation or reversible containment when consequence is high, but it cannot silently become proof of recovery. Chapter 8 owns validity; operations owns timing and response.

Static versus adaptive thresholds

Choose simple, stable rules when the operational boundary has a durable meaning. Seasonal or heterogeneous signals may need adaptive baselines, but only with bounded reference updates, minimum volume, change history, and protection against normalizing a slow regression. Domain owners approve every reference refresh.

Page versus investigation queue

Page for urgent, actionable conditions supported by sufficient evidence, as well as high-consequence prohibited events. Uncertain, delayed, low-volume, and nonurgent drift signals belong in a staffed investigation queue. If the recipient has no credible immediate decision, the rule should not page.

Automatic versus human containment

Narrow, reversible, well-tested actions triggered by high-integrity evidence are good candidates for automation. Human authorization becomes necessary as scope, consequence, uncertainty, statefulness, or irreversibility increases. Automated containment still needs cooldown, expiry, reversal, and an independent safety stop.

Centralized versus domain-owned observability

Centralize contracts, identity, evidence-health controls, and incident protocol while leaving behavioral objectives and response knowledge with the domain. Full centralization loses semantics; full decentralization fragments joins and command. Shared governance defines mandatory fields and interfaces, but domain owners remain accountable for meaning.

Retention versus deletion

Retain evidence only as long as operations, investigation, and required assurance demand. Longer retention applies to defined evidence classes and authorized holds. Deletion must be verified across primary stores, caches, exports, review queues, and incident collaboration; no single store represents the whole lifecycle.

Rapid restoration versus complete diagnosis

When credible harm is ongoing, bounded containment comes before complete root-cause analysis. Preserve evidence and choose a reversible action with explicit verification. Broader or irreversible intervention requires appropriate authority, and restoration leaves the diagnosis and learning obligations open.

9. Production Checklist

Objectives and signals

• User and business outcomes are mapped to behavioral, execution, economic, and evidence-health objectives.
• Each objective declares subject, eligible population, denominator, slices, window, budget or boundary, delay, minimum evidence, missing-data behavior, owner, and response.
• Prohibited events remain explicit and are not averaged into composite health.
• Behavioral signals are classified as invariants, proxies, judgments, feedback, behavior, or delayed outcomes and reference Chapter 8 validity decisions.
• Cost is attributed to useful work and exposes retries, fan-out, fallback, loops, evaluator and tool activity, shared resources, abandoned work, delayed billing, and price identity.
• Drift signals identify subject, reference, slices, expected change, volume, uncertainty, freshness, owner, and action.

Telemetry and evidence health

• The observability contract versions schemas, fields, units, timestamps, causal links, effective-state dimensions, sensitivity, retention, sampling, and unknown semantics.
• Causal identity covers requests, sessions, workflows, queues, attempts, retries, fan-out and fan-in, approvals, actions, state, streaming, cancellation, and delayed outcomes.
• Composite release, configuration, route, prompt, model, context, harness, tool-policy, topology, cohort, and dependency identities are reconstructable where permitted.
• Producer conformance, event loss, duplicates, clock skew, freshness, sampling, joins, cardinality, backpressure, storage, query, dashboard, alert, redaction, retention, and deletion behavior are monitored.
• No-data, late-data, failed-join, and unavailable-evaluator states are explicit.
• Raw payload capture, operational access, export, collaboration, forensic hold, and replay enforce Chapter 9 decisions.

Detection and response

• Every alert protects an objective and defines evidence sufficiency, severity, route, deduplication, allowed automation, test, and review history.
• Incident severity considers consequence, authority, exposure, duration, reversibility, population, ongoing harm, sensitivity, economic loss, and uncertainty.
• Declaration, command roles, scoping, evidence preservation, common timeline, communications, and security handoff are defined.
• Containment covers admission, queues, in-flight work, actions, state, caches, user-visible effects, degraded modes, expiry, reversal, and verification.
• Chapter 10 release controls and Chapter 7 action controls are referenced rather than duplicated.
• Recovery criteria cover infrastructure, behavior, state, external effects, users, evidence health, and delayed outcomes.

Learning and readiness

• Incident reviews analyze impact, detection, response, contributing conditions, control gaps, counterfactuals, and residual uncertainty.
• Actions have owners, due dates, verification, release or policy paths, and closure authority.
• Incident-derived work routes to the correct chapter owner.
• The runbook has been exercised against telemetry loss, semantic degradation, cost amplification, incomplete containment, and delayed recovery evidence.
• Temporary controls and emergency access have expiry and normalization paths.

10. Design Review Questions

1. Which user or business outcome does each operational objective protect, and what prohibited events cannot be budgeted away?
2. What is the exact eligible population, denominator, slice, window, evidence delay, and missing-data behavior for every behavioral and economic signal?
3. Which signals are direct evidence, proxies, sampled judgments, feedback, behaviors, or delayed outcomes, and what does Chapter 8 permit each to support?
4. Can one workflow be reconstructed across queues, retries, fan-out, approvals, actions, state transitions, release changes, and delayed outcomes?
5. Which effective release, configuration, route, prompt, model, context, harness, tool-policy, topology, and dependency identities are recorded at each step?
6. How is telemetry loss distinguished from zero failures, and what happens when critical evidence becomes indeterminate?
7. What useful unit does cost describe, and how are retries, loops, shared resources, abandoned work, and delayed billing attributed?
8. What reference and expected changes make a drift signal interpretable, and who may refresh the reference?
9. For each page, what urgent decision is expected, what evidence is required, and how are duplicate or conflicting rules controlled?
10. What consequence, authority, exposure, duration, reversibility, sensitivity, and uncertainty determine incident severity?
11. Which containment action is narrowest, who may authorize it, what queued or in-flight work survives it, when does it expire, and how is its effect verified?
12. What evidence can be preserved when raw prompts, context, or outputs cannot, and where may incident collaborators access it?
13. Which technical, behavioral, state, action, user, telemetry, and delayed-outcome claims must be true before recovery is declared?
14. Which follow-up would actually break the failure path, how will it be verified, and who can accept the residual risk if it is not implemented?

11. Main Artifact: AI Operations Runbook

The AI Operations Runbook is one living control record with ten coordinated views. It references the Evaluation Plan, AI Threat Model, AI Release Manifest, and Tool Permission Matrix rather than reproducing them.

View 1: Service charter and operating boundary

The operational handoff is monotonic. Operators may contain, narrow, suspend, or move the service into an approved degraded mode, but they may not silently redefine product eligibility, action authority, evaluation validity, governance constraints, or release identity. Emergency authority is explicit, scoped, expiring, and routed back to the owning chapter's decision record.

View 2: Operational objective catalog

Record hard-stop prohibited events separately from budgeted objectives. Link behavioral objectives to the Chapter 8 evidence-validity record.

View 3: Telemetry and causal-identity contract

View 4: Dashboard and investigation map

Include execution, behavior, economics, drift, dependencies, action state, effective runtime state, and telemetry health. Store query versions and result-snapshot procedure.

View 5: Detection and alert catalog

Document symptom versus suspected cause, related evidence-health rules, missed-incident review, and stale-rule retirement.

View 6: Incident classification and command plan

View 7: Containment and degraded-mode matrix

Cover observation, increased sampling, feature or cohort constraint, review-only mode, admission stop, queue pause, state isolation, Chapter 10 transitions, and Chapter 7 cancellation or reconciliation.

View 8: Evidence and investigation protocol

View 9: Recovery and remediation plan

Reference Chapter 10 recovery receipts and Chapter 7 action reconciliation. Record who may accept an explicitly degraded mode or residual uncertainty.

View 10: Learning and improvement record

The runbook is ready when an operator can move from a signal to evidence validation, declaration, scoped containment, protected investigation, verified recovery, communication, and durable learning without inventing authority or semantics during the incident.

12. Key Takeaways

• AI operations protects user and business outcomes through behavioral, execution, economic, and evidence-health objectives.
• An operational signal is trustworthy only when its population, denominator, identity, delay, sampling, and missing-data behavior are explicit.
• Causal execution graphs must follow durable workflows, queues, approvals, actions, state, and delayed outcomes—not only synchronous requests.
• Cost, drift, and quality signals require contracts; raw tokens, distribution shifts, and proxies are not conclusions.
• The observability system is itself a production dependency. Missing evidence creates an indeterminate state, not a healthy one.
• Incident response requires command, bounded authority, protected evidence, and containment that accounts for active work and external effects.
• Recovery is proven across infrastructure, behavior, state, actions, users, telemetry, and delayed outcomes.
• Incidents close when risk reduction is verified or residual risk is explicitly accepted—not when local patches and tickets are complete.