Delivery Foundations

Trusted Increments and Acceptance Evidence

Learn how acceptance criteria and evidence support an accountable decision that the delivered result matches agreed intent, satisfies applicable requirements, and leaves current-state knowledge accurate.

Authors: Marcus Peck

Published
Last updated
Cite this paper

A delivery increment reaches completion only when the organization can accept the resulting product state. The implementation may compile, the test suite may pass, and an AI coding agent may report that its work is finished, but they cover only the conditions they actually test or inspect.

Requirements, Structured Discussion, and Increments defines an increment as one complete delivery iteration for a coherent product change. It begins with a requirement, develops through discussion and the applicable specifications, and ends with implementation, verification, acceptance, and shipment. This paper focuses on that final part of the lifecycle: how a completed implementation acquires enough evidence and review to become a trusted increment.

Testing is a central source of acceptance evidence, including unit testing, integration testing, user acceptance testing, and stress testing. Specification-First Delivery places those results within a broader acceptance decision.

The acceptance basis starts well before verification. Product, architecture, security, reliability, data, quality, engineering, and other relevant functions establish conditions within their authority as the Specification Set develops. Technical specifications make those conditions concrete enough to implement. Test specifications describe how suitable conditions will be exercised, measured, or inspected. The condition should remain recognizable throughout that process even though its form becomes more specific.

A six-company case study of 30 practitioners found that weak communication of requirements changes to testers could leave new requirements unverified and obsolete requirements incorrectly verified, contributing to software-quality problems, wasted effort, and delays.1 This is especially relevant to AI-based applications. Certain situations are verifiable in a deterministic way through unit or integration testing. In other cases, the behavior of the model may need an annotated data set, evaluation metrics, iterations, or human sampling. Other factors like architecture, code design, security mechanisms, functionality, efficiency, and maintenance might require separate verification methods.

Three concepts organize the final part of the increment.

Verification

Produces and examines evidence for a defined condition. It may run tests, inspect dependencies, evaluate model outputs, compare persisted state, review architecture, measure latency, or calculate provider cost.

A trusted increment is an accepted increment for which the organization has sufficient, reviewable evidence that the resulting product state conforms to the reviewed intent and applicable requirements, and for which material knowledge produced during delivery has converged into a usable current state.

Trust here means that the organization can explain and defend its decision about the current product state, including the evidence it relied on and the limitations it still recognizes. Defect evidence remains important, but it is one part of a broader acceptance basis.

Back to top

1. Acceptance Criteria Within the Specification Set

An acceptance criterion is a condition that must be satisfied for an applicable requirement, specification, or delivery result to be accepted. In simple work, the criteria may fit comfortably at the bottom of a user story or engineering ticket. A material change usually has a wider acceptance basis.

Product behavior may be only one part of the decision. The increment may also need to satisfy architecture, security, reliability, privacy, data semantics, compatibility, performance, supportability, observability, accessibility, cost, or repository-wide engineering requirements. Those conditions originate with different functions because they represent different kinds of professional judgment.

The Specification Set carries those judgments into delivery.

  1. Product and functional intent

    Standing product requirements and functional specifications define the outcomes, supported behavior, exclusions, exceptions, and product rules that the increment must preserve or introduce.

    • Intended user and system outcomes
    • Unsupported or prohibited behavior
    • Business rules and exception paths
  2. Architecture and specialized requirements

    Architecture, security, reliability, data, integration, infrastructure, accessibility, and other specialized specifications contribute conditions within their professional authority.

    • System responsibilities and interfaces
    • Security and authorization requirements
    • Reliability and failure behavior
    • Data, integration, and operational constraints
  3. Governing instructions

    Repository-wide engineering rules may form part of acceptance when they establish required code structure, implementation practice, test discipline, documentation, or prohibited shortcuts.

  4. Technical specifications

    Technical specifications translate reviewed upstream decisions into implementation responsibilities, technical constraints, protected behavior, and evidence obligations.

  5. Test and QA specifications

    Test and QA specifications define scenarios, datasets, expected results, measurements, inspections, and evidence collection without acquiring authority to redefine the requirement being tested.

The source of a criterion continues to matter after it reaches engineering. A security requirement does not become an engineering preference because a technical specification has to implement it. A functional outcome does not become a QA decision because QA needs a way to test it. A reliability condition does not disappear because the happy path works. When verification exposes a missing product, architecture, security, reliability, or other material decision, the unresolved question should return to the role that has authority over that subject. Verification should make an earlier decision testable, not quietly replace it with a new one.

Back to top

1.1. Running Example: A Personal Day Planner

Consider a personal day planner with a calendar and an AI planner. The planner accepts requests such as:

Move my gym session to after my 4:00 PM meeting.

or:

Find an open hour tomorrow afternoon and schedule project review time.

The product allows the planner to inspect the user's calendar, propose or execute supported changes, and explain the result. Its purpose is deliberately narrow: personal calendar planning rather than general-purpose assistance.

A plausible implementation uses an LLM to interpret the request, server-side logic to determine which actions are allowed, domain services to read and mutate calendar state, and a client application to stream progress and display changes. The exact model provider or agent framework is not important to the acceptance model.

One increment that extends this planner could be subject to all of the following conditions:

SourceExample acceptance condition
Product or AI behavior requirementThe AI planner may assist only with personal calendar-planning requests.
Functional specificationSupported requests produce the agreed calendar outcome; unsupported requests do not execute calendar actions.
Architecture specificationThe LLM may propose actions, while server-owned controls determine whether a calendar mutation may execute.
Security specificationA user may mutate only calendar resources that the user is authorized to modify.
Reliability specificationIf the AI provider or planning workflow fails, existing calendar data and confirmed changes must remain intact, manual calendar editing must remain available, and the failed planning attempt must be observable to operators.
Operational requirementPlanner failures and blocked actions must be observable through defined logs or traces.
Performance requirementA defined class of ordinary planner turns must meet the agreed response-latency objective.
Cost requirement, if establishedModel and provider cost for the defined acceptance workload must remain within the agreed threshold.
Governing instructionsDomain logic, tests, documentation, and code structure must follow repository-wide implementation rules.

Some conditions describe visible behavior. Others describe system structure or decision authority. Reliability and security may define invariants that must hold even when the model makes a poor judgment. Performance and cost are meaningful only when the organization has established the workload and threshold that matter.

The example also shows direct delivery participation. Product defines the planner's purpose and expected outcomes. Architecture places system responsibilities and control points. Security defines which mutations are permitted. Reliability defines retry and failure behavior. Engineering turns those decisions into executable work. Quality specialists choose verification methods that can expose a failure. Acceptance depends on the combination rather than on one downstream checklist.

Back to top

2. Transformation of Acceptance Criteria Across Specification Levels

Acceptance criteria become more specific as delivery approaches implementation. Their representation changes because each specification has a different job to do, but the underlying decision should remain intact.

Acceptance criterion transformation

From authoritative condition to acceptance evidence

  1. 1

    Authoritative condition

    A standing requirement or specialized specification establishes the condition at the level of the accountable product or professional decision.

  2. 2

    Functional or specialized expression

    The condition is stated in terms of observable behavior, professional constraints, required outcomes, prohibited outcomes, or operating expectations.

  3. 3

    Architecture and control expression

    Where needed, architecture or another specialized specification determines which system responsibilities, interfaces, controls, and decision points must preserve the condition.

  4. 4

    Technical implementation responsibility

    The technical specification assigns concrete implementation work and constraints without weakening or reinterpreting the upstream decision.

  5. 5

    Test and QA verification

    Verification specifications define suitable scenarios, datasets, metrics, inspections, or reviews that can produce evidence for the condition.

  6. 6

    Acceptance evidence

    The resulting test results, measurements, reviews, records, and observations are traced back to the condition for conformance review and acceptance.

With the same philosophy mentioned in Requirements, Structured Discussion, and Increments, the complete chain is not mandatory for every criterion. A deterministic business rule may move from a functional specification directly into one technical specification and a unit test. An AI capability that changes system behavior, authorization, and operational risk may need several specialized specifications before engineering has enough information to implement it safely.

Back to top

2.1. Example: AI product scope and calendar mutation

Suppose the standing AI requirement is that the planner may assist only with personal calendar-planning requests. That one condition takes a different form at each specification level.

Standing AI requirement

The AI planner may assist only with personal calendar-planning requests. This establishes product scope without prescribing a classifier, prompt, model provider, or module structure.

This separates two problems. The model must understand a request well enough to classify or interpret it. The product must also ensure that a poor classification cannot silently become an unauthorized calendar write. Semantic interpretation may be probabilistic; mutation authority should be protected by explicit system controls wherever practical.

The QA or test specification may use a labeled corpus containing clear in-domain requests, clear out-of-domain requests, ambiguous requests, mixed-intent requests, adversarial attempts, and follow-up turns whose meaning depends on earlier calendar context. It can then define evaluator metrics and release thresholds such as:

Calendar-scope evaluation corpus:
- Critical out-of-domain cases producing mutation authority: 0
- In-domain recognition: >= agreed acceptance threshold
- Required clarification behavior on labeled ambiguous cases: >= agreed acceptance threshold
- Mixed-intent cases executing unsupported actions: 0

Thresholds belong to the acceptance design. They should reflect the product's intended behavior and risk tolerance, not be adjusted afterward to fit the score achieved by the current implementation.

A model score alone is not enough to establish the execution invariant. Blocked requests can be run against a known calendar state and verified to produce zero mutations. Architecture review can confirm that authorization remains server-owned. Tool-call or audit records can show that the blocked request never materialized a calendar write.

Semantic evidence

Evaluator results show how well the planner interprets calendar-planning scope over the defined corpus and configuration. They support a statistical claim about model behavior.

The two evidence forms answer different questions and should remain separate in the acceptance record.

Back to top

2.2. Example: Reliability requirement and retry safety

The same transformation is simpler when the requirement is deterministic. A reliability specification may state:

Retrying the same accepted calendar mutation must not create duplicate calendar items.

The technical specification can assign an idempotency mechanism to the calendar mutation service and define how a repeated request is identified. The test specification can then exercise the guarantee directly:

1. Start from a known calendar state.
2. Submit one authorized request to create a calendar item using request identifier X.
3. Retry the same mutation using request identifier X.
4. Read the resulting calendar state.
5. Verify that exactly one corresponding item exists.
6. Verify that the retry result follows the agreed response contract.

Here, the evidence path is straightforward: the reliability requirement leads to a technical responsibility, an integration scenario, and a persisted-state comparison. The form differs from the AI scope example, but the original criterion remains traceable while downstream specifications make it executable and verifiable.

Back to top

3. Verification Methods and Acceptance Evidence

Acceptance evidence is the material used to decide whether applicable criteria were met. The method should match the claim. A unit test can give strong evidence about a deterministic function-level rule and little evidence about dependency direction. A screenshot can show a visible UI state but says nothing about authorization. A model-evaluation score can describe behavior over a corpus but cannot prove that a server-side control is impossible to bypass. A successful build proves that compilation succeeded, not that the intended product behavior exists.

Structured assurance cases connect claims to evidence through explicit arguments. The model requires evidence to be traceable to its source and method of origination; examples include test results, formal analyses, simulations, inspections, and deterministic, probabilistic, or qualitative information.2

The practical rule is to use evidence that directly examines the condition being accepted and to prefer stronger, more independent evidence when the consequence of failure is high.

  1. Automated behavioral verification

    Use deterministic tests when behavior can be expressed as repeatable inputs, state transitions, and expected outcomes.

  2. AI evaluation

    Use defined corpora, evaluators, thresholds, repeated trials, and failure analysis when the claim concerns model behavior over a distribution of inputs.

  3. Structural and architectural verification

    Inspect dependencies, interfaces, network paths, schemas, and responsibility placement when the requirement concerns system structure rather than output alone.

  4. Static quality and code structure

    Use static checks and focused code review when governing instructions or technical specifications establish implementation-quality conditions.

  5. Security reliability and operations

    Combine protection tests, failure scenarios, state comparisons, audit records, logs, and traces when the acceptance condition concerns authorization, recovery, or operational behavior.

  6. Performance capacity and cost

    Use controlled workloads and defined measurement bases when latency, throughput, resource consumption, or provider cost is part of the acceptance basis.

  7. Professional review

    Use qualified human judgment for material conditions that cannot be reduced adequately to one automated signal.

  8. Knowledge consistency

    Verify that durable specifications, documentation, instructions, and other current-state knowledge still describe the product that is being accepted.

Back to top

3.1. Automated Behavioral Verification

Unit, integration, end-to-end, regression, contract, and property-based tests are strong evidence when the condition can be expressed as a repeatable relationship between inputs, system state, and expected outcomes. Their main strength is not simply automation. It is that the same claim can be exercised consistently as the implementation changes.

For the personal day planner, automated verification can cover:

  • canonical time parsing and validation;

  • calendar state transitions after add, move, update, and delete operations;

  • authorization results for permitted and prohibited calendar mutations;

  • idempotent retries and duplicate prevention;

  • zero state mutation after a request has been classified as blocked;

  • rollback, cancellation, and undo semantics;

  • API and serialization contracts between the planner, calendar service, and client;

  • timeout and exception behavior; and

  • preservation of existing manual calendar behavior when the AI planner changes.

Granularity in testing is important. Unit testing can work for deterministic parsing rules. But the same unit test will not be able to prove that a blocked request will still be non-mutating after all the steps of interpreting the model, server authorization, picking the right tool, persisting the object, and publishing the event. Proof of that assertion would need integration through components responsible for that invariant.

Regression tests also carry a specific acceptance role when an upstream specification identifies behavior that must remain unchanged. In that case, the regression test is not merely a historical guard against accidental breakage. It is evidence for a protected behavior that belongs to the current increment's acceptance basis.

Tests are most useful when their expected result traces to a reviewed condition. An executor can generate large amounts of test coverage around its own implementation choices, but coverage of internal details does not establish that the organization asked for the right behavior in the first place.

Back to top

3.2. AI Evaluation

LLM acceptance evidence needs an evaluation design that makes model behavior measurable against the product conditions under review. The design can define representative scenarios, a versioned corpus, metrics or rubrics, model and configuration details, thresholds, repeated trials where appropriate, failure classes, and records that allow a reviewer to understand how the result was produced.

HELM offers a useful reference point for structuring this kind of evidence. It evaluates language models across 16 core scenarios and seven metrics, supplemented by targeted evaluations of specific capabilities and risks.3 By separating scenarios from the metrics used to assess them, it shows how model evaluation can expose several dimensions of behavior without collapsing them into one score. In Specification-First Delivery, those evaluation choices sit inside the acceptance chain: requirements and specifications establish which product behaviors matter, the evaluation design turns suitable behaviors into measurable evidence, and conformance review uses that evidence as part of an accountable acceptance decision.

The same pattern can be implemented within existing product and test infrastructure. A versioned acceptance corpus can live in the repository; ordinary test code or scripts can execute model calls and calculate agreed metrics; documented human-review rubrics can cover judgments that resist simple scoring; and the resulting records can be retained as acceptance evidence. A dedicated language-model evaluation framework such as HELM can improve repeatability, comparability, or evidence management. The required capability is the evaluation design and evidence trail, whether implemented with a dedicated framework or existing engineering infrastructure.

For the planner, AI evaluation may examine whether the model:

  • recognizes requests that are clearly within the calendar-planning domain;

  • rejects or redirects clearly out-of-domain requests;

  • asks for clarification when a request is genuinely ambiguous;

  • produces valid structured planning output for downstream execution;

  • preserves relevant information across conversational follow-ups;

  • keeps user-facing explanations consistent with actions actually executed; and

  • meets an agreed quality rubric for recommendations where there is no single correct schedule.

A result such as 93% has little meaning without its measurement basis. Where evaluator evidence contributes to acceptance, the delivery record should make the corpus or corpus version, model and material configuration, evaluator method, threshold, and significant failure classes discoverable.

If an LLM is used as an evaluator, the evaluator becomes part of the measurement system. Zheng et al. identified position, verbosity, and self-enhancement biases, as well as limited reasoning ability, in LLM-as-a-judge evaluation.4 Repeated measurement may also be needed when model sampling introduces material variation.

The acceptance threshold is part of the requirement or evaluation design, not a number to be chosen after seeing the current model's score. Teams may revise a threshold when new evidence shows that the original standard was inappropriate, but that is a specification decision and should be reviewed as such.

Probabilistic evidence should remain probabilistic in the acceptance record. It supports a claim about observed performance over a defined population and configuration. It does not create a guarantee about every future input.

Back to top

3.3. Structural and Architectural Verification

Some acceptance conditions concern where responsibility resides, which components may communicate, which source is authoritative, or which control must be crossed before an action can occur. These are architectural claims, and runtime output alone may not reveal whether the implementation satisfies them.

The software-architecture literature includes static compliance-checking techniques. Knodel and Popescu compare reflexion models, relation conformance rules, and component access rules across 13 applicability dimensions.5

Suppose the architecture specification for the planner requires:

The client must not call the model provider directly.

Evidence for that condition can include:

  • browser or client network inspection showing that AI requests go only to application-server endpoints;

  • dependency inspection confirming that model-provider adapters are server-side;

  • confirmation that provider credentials are absent from client bundles, browser storage, and client configuration;

  • route and service inspection showing that server-side authorization occurs before model-initiated mutation; and

  • architecture review confirming that no alternate path bypasses the intended control point.

Similarly, if the architecture says that calendar mutation belongs to a calendar domain service, verification should inspect whether routes, agents, or UI handlers write directly to persistence or reimplement domain rules elsewhere. A happy-path integration test may still pass even when the responsibility has moved to the wrong layer.

Useful techniques include dependency-graph analysis, import inspection, schema and interface comparison, network inspection, route tracing, source-of-truth review, and focused inspection of the code paths that enforce architectural decisions.

Architecture conformance is especially important in AI-assisted implementation because an executor can produce behavior that appears correct while introducing a shortcut that weakens a control point. The resulting system may pass functional tests and still be structurally non-conformant.

Back to top

3.4. Static Quality and Code-Structure Verification

Governing instructions and technical specifications may establish code-quality or implementation-structure conditions that are material to acceptance. These conditions need an explicit basis. They should not depend on a reviewer deciding at the end that they dislike the shape of the code.

Evidence can include:

  • lint and typecheck results;

  • static security or correctness analysis;

  • dependency and import-rule checks;

  • duplication or complexity inspection where those measures are meaningful;

  • module and service ownership review;

  • naming and contract consistency;

  • confirmation that domain behavior remains in the intended service layer;

  • confirmation that shared types or schemas are used instead of being redefined locally;

  • confirmation that prohibited hardcoded routing, fallback logic, or special cases were not introduced; and

  • review of technical debt that the increment would otherwise make part of the accepted product state.

For example, repository governing instructions may require route handlers to remain declarative while services own business logic. A technically functioning change that embeds calendar business rules directly in an HTTP route can violate that rule even if every endpoint test passes.

Static checks are valuable because they can make some structural expectations repeatable. They still do not replace professional review when the question is whether an abstraction remains understandable, responsibilities are sensibly separated, or the design creates a maintenance burden that no simple metric captures.

When a code-quality condition can block acceptance, its source should be discoverable in governing instructions, architecture, the technical specification, or an established review policy. That keeps acceptance tied to known engineering standards rather than personal taste.

Back to top

3.5. Security, Reliability, and Operational Verification

Security and reliability requirements often describe what must happen when the normal path is attacked, interrupted, repeated, partially completed, or unavailable. Their evidence therefore needs to exercise protection and failure behavior, not only successful execution.

For the planner, relevant evidence may include:

  • cross-user or cross-calendar mutation attempts that demonstrate authorization enforcement;

  • tool-call and payload validation, including schema rejection and unsupported fields;

  • retry scenarios that verify idempotency and duplicate prevention;

  • cancellation and rollback after partial execution;

  • timeout behavior across the model, tool, and persistence path;

  • controlled provider or dependency failures;

  • confirmation that a failed action cannot be reported to the user as successfully completed;

  • audit records that identify blocked and executed actions; and

  • logs or traces showing that failed, retried, recovered, or blocked operations remain observable.

A security specification may also require evidence that a particular secret never reaches the client, that user identity is revalidated at the mutation point, or that tool permissions are narrower than the set of actions the model can propose. These claims may require a mixture of static inspection and runtime scenarios.

Operational evidence answers a different question from code inspection. A code review may confirm that telemetry calls exist. A controlled failure run can show whether those calls actually produce enough information to diagnose the event. When observability is itself an acceptance condition, both implementation and resulting records may matter.

Back to top

3.6. Performance, Capacity, and Cost Verification

Performance and economic constraints become acceptance conditions when they are part of the applicable specification set. They should be measured against a defined workload rather than treated as vague expectations such as "fast enough" or "cheap enough."

A planner increment might define:

For the defined ordinary-turn workload, p95 time to first user-visible progress event must remain below the agreed threshold.

or:

For the defined acceptance corpus and model configuration, average provider cost per completed planning turn must not exceed the agreed threshold.

Evidence may include controlled performance runs, token and model usage records, provider-cost calculations, infrastructure measurements, concurrency tests, percentile analysis, or comparisons with an agreed baseline.

The measurement context must be clear and be part of the evidence that is shipped with the delivery. A p95 figure without the environment, workload, data shape, and concurrency assumptions is difficult to interpret. A provider-cost figure without the model, pricing basis, tool usage, retries, and acceptance workload can be equally misleading.

Cost should not become a surprise acceptance criterion after implementation. If a cost limit was never established, a reviewer should not invent one simply because the measured result looks high. If cost was specified upstream, however, functional correctness does not excuse the team from measuring it.

Back to top

3.7. Human Professional Review

Some material conditions remain best evaluated through qualified professional judgment. Architecture coherence, maintainability, migration risk, threat implications, accessibility quality, usability, and the suitability of an abstraction may resist reduction to one automated check.

Professional review becomes useful acceptance evidence when the subject and conclusion are explicit. A record such as "architecture reviewed" is weak if nobody can tell which decision was examined. A stronger review identifies the applicable condition, the reviewer or responsible role, the material observations, and the resulting conclusion or required follow-up.

The professional review should still be directed at a defined acceptance concern. Specification-First Delivery preserves professional judgment by making its role and authority visible. When code is developed with AI coding tools, professional review focuses on the verification evidence, conformance review, and acceptance decision. Line-by-line code review may contribute where appropriate, alongside the other evidence needed for the conditions under review.

Back to top

3.8. Knowledge Consistency Verification

The delivered code is only one part of the accepted product state. The Shared Knowledge System must remain accurate enough for the next qualified participant to understand, operate, and change the product without reconstructing the accepted state from old conversations and commit history.

One repository-scale study focused specifically on outdated code-element references in README and wiki documentation. In its top-1000 GitHub dataset, 28.9% of projects contained at least one outdated code-element reference at the time of analysis; among 800 projects from that dataset whose full history was examined, 82.3% had contained one at some point in their history.6

Verification should therefore ask whether the increment changed information represented in:

  • current architecture documentation;

  • standing product requirements;

  • functional or specialized specifications that remain authoritative after delivery;

  • interface contracts and data-semantics documentation;

  • governing instructions;

  • operational and support guidance;

  • README material and configuration examples;

  • test explanations or acceptance datasets that future work will reuse; and

  • inline comments whose meaning is material to later implementation.

Not every document mentioned during an increment needs to be updated. Historical delivery records can remain historical. The important question is whether the sources that claim to describe the current product still describe the state being accepted.

For the day planner, an architecture change that moves mutation authorization into a new server service should update the authoritative architecture material if future engineers are expected to rely on it. A new retry guarantee may need to appear in the relevant reliability or interface documentation. If a previously supported behavior has become intentionally unsupported, the standing product or functional knowledge should not continue to describe the old behavior as current.

Knowledge Convergence Across the Increment Lifecycle describes how implementation learning and competing professional judgments acquire scope and authority. Knowledge consistency verification is the acceptance-time check that those updates have actually reached the durable sources that future delivery will use.

If the code and current-state knowledge disagree, the increment leaves two competing descriptions of the product. That is a conformance problem even when the software itself works.

Back to top

3.9. Relationship to Test-First Development

Test-Driven Development and related test-first practices provide a useful discipline: express an expected result before writing the implementation that satisfies it. When a reviewed acceptance condition can be represented well as a test, writing that test first can constrain both human and AI executors.

In two experiments with university students, adding Fit acceptance tables to textual requirements improved requirement understanding without a significant increase in comprehension effort.7

Acceptance design can begin before implementation even when the evidence is not an automated test. The team should consider what evidence will support acceptance whenever ambiguity, risk, dependency, or consequence makes that worthwhile. The evidence might be an automated test, but it could also be an evaluator, architecture inspection, performance measurement, cost calculation, or professional review.

AI makes the relationship more important because it can generate tests as quickly as it generates code. An ambiguous requirement can lead the same agent to choose an interpretation, write tests for that interpretation, implement code that passes them, and report success. The result may be internally consistent and still reflect a decision that nobody with the relevant authority made.

The acceptance basis therefore needs to come from reviewed intent and specifications rather than from the executor's ability to produce a self-consistent implementation and test suite.

Back to top

4. Conformance Review of the Delivered Increment

Verification examines particular conditions. Conformance review assembles those results and asks whether the integrated increment matches the applicable Specification Set. A fully green test run may still leave a gap if an architecture requirement was never inspected, a cost threshold was not measured, a standing security condition has no implementation owner, or current-state knowledge still describes the previous product state.

For a material increment, the review commonly covers several dimensions.

  1. Behavioral and functional conformance

    Confirm required user and system outcomes, prohibited outcomes, important exception and failure paths, state transitions, preserved behavior, regressions, and compatibility obligations.

    • Supported scheduling requests
    • Out-of-scope non-mutation
    • Ambiguous time handling
    • Rollback and manual calendar compatibility
  2. Architectural and implementation conformance

    Confirm system responsibilities, interfaces, dependency direction, source-of-truth decisions, control points, code structure, governing instructions, and prohibited implementation paths.

    • Client routes AI work through the application server
    • Model interpretation remains separate from server-owned authorization
    • Calendar mutation remains in the intended domain service
    • No fallback path bypasses reviewed controls
  3. Security reliability operations performance and cost

    Connect specialist requirements to authorization, validation, failure handling, retry, rollback, recovery, auditability, observability, performance, capacity, and cost evidence when those concerns apply.

  4. Knowledge and delivery records

    Confirm that material implementation learning has been resolved or given explicit status, current-state knowledge is updated, and delivery records preserve the deviations, decisions, and evidence that may matter later.

The review follows the applicable obligations, not merely the checks that happened to run. This is what makes missing evidence visible. It also prevents one strong evidence category from masking a gap in another. A broad integration suite cannot compensate for an unreviewed architecture deviation if architecture conformance was part of the acceptance basis.

Back to top

5. Evidence Traceability and Acceptance Coverage

Evidence has little value if the team cannot tell what it proves. Traceability connects each material acceptance condition to its source, implementation responsibility, verification method, evidence, and final status.

An embedded driver-assistance study linked 443 annotated natural-language requirements to 1,300 simulation executions and 53 test-drive executions, allowing the researchers to compare how well test stages and test cases aligned with the requirements.8

The framework does not require a universal traceability-matrix document. A team may represent these links in specifications, test records, issue systems, generated reports, or another form that suits its delivery tooling. A qualified participant should be able to recover the relationship for material conditions.

For the retry requirement:

FieldExample
SourceReliability requirement R-08
Acceptance criterionRetrying the same accepted mutation must not create a duplicate calendar item.
Technical responsibilityCalendar mutation service and idempotency mechanism
Verification methodIntegration test with repeated request identifier and persisted-state comparison
EvidenceTest result plus resulting calendar-state record
Conformance statusSatisfied

The AI scope requirement uses several evidence sources:

FieldExample
SourceStanding AI behavior requirement AI-03
Acceptance criterionOut-of-domain requests must not execute calendar actions.
Technical responsibilityScope evaluation, server authorization, and calendar mutation gate
Verification methodLabeled AI evaluation corpus, blocked-request integration scenarios, architecture inspection, audit review
EvidenceEvaluator metrics, zero-mutation state comparisons, architecture review result, tool-call audit records
Conformance statusSatisfied, rejected, or returned for amendment based on the defined thresholds and invariants

The above associations form what is called acceptance coverage. Coverage means that the company has the capability of identifying, for each one of the important acceptance criteria, the existence of an implementation approach and the existence of an adequate assessment approach. This definition varies from traditional software engineering since in software engineering, coverage is usually defined as the coverage of source code through testing. It is indeed an important parameter but only a part of acceptance coverage.

The incomplete acceptance coverage is called coverage gap. A coverage gap appears when a material acceptance condition has no complete path from requirement to evidence. Suppose the security specification states:

A user must not be able to modify another user's calendar.

That requirement needs more than a statement in the specification. The delivery record should make it possible to identify who enforces the rule, how the rule is verified, and what evidence demonstrates that it held.

100%
Acceptance coverage for cross-user calendar mutation
The security requirement is carried into a technical responsibility, exercised through a cross-user mutation scenario, and supported by rejection, unchanged-state, and audit evidence before conformance can be judged.

If the chain stops earlier, the acceptance coverage is incomplete. For example, the requirement may exist without any component being assigned responsibility for enforcing it, or the implementation may contain an authorization check without any verification scenario that demonstrates the protection works across the full mutation path.

This is why the number of passing tests is not a useful measure of acceptance coverage by itself. Thousands of green tests elsewhere in the system do not establish that cross-user calendar mutation is prevented unless some of the resulting evidence directly addresses that claim.

Even within conventional testing, code coverage is an incomplete proxy for test effectiveness. Across 31,000 test suites for five large Java systems, Inozemtseva and Holmes found only low-to-moderate correlation between coverage and fault-detection effectiveness after controlling for test-suite size, and concluded that coverage should not be used as a quality target.9

The reverse is also possible. An increment may have extensive linting, unit tests, code coverage, performance checks, and other useful engineering controls while still leaving a material requirement without evidence. Acceptance coverage is therefore about whether each material acceptance condition has an implementation path and suitable evidence, not about how many checks were executed.

Back to top

5.1. Probabilistic Evidence for AI Behavior

AI evaluation changes the shape of some acceptance evidence because model quality is often statistical. The acceptance record should distinguish the kind of claim supported by each form of evidence.

Deterministic evidence

Uses repeatable inputs, states, and expected outcomes to establish a specific behavior or invariant. Examples include authorization checks, persisted-state comparisons, idempotency tests, schema validation, and rollback behavior.

For the day planner, a scope-classification evaluator can show performance over a representative corpus. It cannot prove that every future request will be classified correctly. Critical product invariants should therefore avoid depending on one probabilistic result where architecture and runtime controls can protect them more directly.

Different conditions may use different measurement forms: unauthorized mutation may require zero observed violations together with deterministic controls; classification quality may use precision, recall, or false-negative rate; recommendation quality may use an evaluator rubric and sampled review; latency may use p95 or p99; cost may use mean, percentile, or total workload cost; reliability may combine deterministic fault scenarios with operational observations.

Evidence can itself be stale, weak, or misleading. The reviewer should be able to determine whether it came from the implementation and configuration proposed for acceptance, whether the test or corpus represents the condition being measured, whether significant negative and exception cases are present, and whether material configuration is identifiable. Thresholds should not move simply because the implementation missed them.

Back to top

6. Accountable Acceptance and Trusted Increment State

Acceptance closes the delivery iteration as an owned product-state change. Organizations may combine acceptance with merge approval, release approval, or deployment workflow, but the underlying decision remains distinct: are the relevant accountable participants prepared to accept this state on the evidence available?

No universal role has to accept every dimension. Decision rights follow the organization's operating model and the Specification Set. A Product Owner may own product behavior, an architect a material architecture decision, security specialists required controls, engineering technical conformance, and a service or release owner deployment readiness. The relevant authority should be known rather than silently inherited by the executor or automated pipeline.

AI can support almost every preparation step. It can run tests, execute evaluator suites, inspect dependencies, calculate cost, compare implementation with specifications, summarize evidence, and identify missing coverage. It can also expose contradictions that require a specification to be revisited. Those capabilities do not give the AI organizational decision rights over acceptance.

Back to top

6.1. Failed or Insufficient Evidence

A failed condition should be traced to the layer that needs to change.

  1. Implementation defect

    The implementation does not satisfy a confirmed technical specification. Correct the implementation and regenerate the affected evidence.

  2. Technical specification problem

    The technical specification cannot realize an upstream condition or assigned the wrong implementation responsibility. Revise it under the appropriate engineering and specialist review.

  3. Verification method problem

    The test, evaluator, dataset, inspection, or measurement does not actually examine the intended acceptance criterion. Revise the test specification or verification method.

  4. Architecture decision problem

    The reviewed architecture proves insufficient, contradictory, or unable to preserve the required behavior or control. Return the material decision to the appropriate architecture authority.

  5. Upstream requirement problem

    A functional, standing, security, reliability, or other authoritative requirement is incomplete or incorrect. Return the decision to the role that owns that requirement.

  6. Knowledge convergence problem

    Implementation or verification produced material new knowledge that changes shared understanding. Resolve its authority and update affected specifications or current-state knowledge before acceptance.

Verification can legitimately change the increment because it may expose information that was unavailable earlier. What it should not do is change the acceptance standard silently. An executor should not lower a threshold, rewrite a test around current behavior, bypass an architectural control, or reclassify a required path as out of scope simply to obtain a pass.

Back to top

6.2. Acceptance Outcomes

A conformance review normally leads to one of three governed outcomes.

Accept

The applicable conditions have sufficient evidence, material conformance gaps are resolved, required knowledge has converged, and the relevant accountable participants accept the resulting state.

Specification-First Delivery does not prescribe one approval workflow. It requires the basis of the decision, the relevant authority, and unresolved conditions to remain discoverable.

Back to top

6.3. Proportionality

Acceptance evidence should be proportionate to ambiguity, novelty, dependency, risk, and consequence.

  1. Small local change

    A one-line typo or similarly low-risk local correction may need only focused review and the repository's normal checks.

  2. Deterministic bug fix

    A contained bug fix may need a reproducing test, corrected implementation, relevant regression checks, and confirmation that no durable knowledge became inaccurate.

  3. Cross-system AI increment

    A change involving probabilistic interpretation, mutation authority, security, retries, UI integration, telemetry, and provider cost may justify several specialized specifications, evaluator datasets, architecture review, integration tests, and performance or cost evidence.

The aim is enough evidence for the decision the organization actually has to make. Too little evidence leaves material conditions unexamined. Too much process around a low-risk change adds cost without improving the acceptance judgment.

Back to top

7. Operating Outcome

A trusted increment brings together the decisions made throughout the delivery cycle. Requirements and standing knowledge establish what the product needs. Specialized specifications contribute the professional conditions that apply. Technical specifications assign executable responsibility. Test specifications define suitable verification. Implementation produces the change and the evidence-producing surfaces. Conformance review checks the integrated result. Accountable participants decide whether to accept it.

Trusted increment

Conditions for an accepted product state

  1. 1

    Reviewed requirements and specifications

    The organization can identify the intent, professional conditions, authority, constraints, and acceptance criteria that applied to the increment.

  2. 2

    Conformant implementation

    The delivered code, configuration, architecture, and operational behavior implement the reviewed decisions without unresolved material deviation.

  3. 3

    Sufficient acceptance evidence

    Tests, evaluator metrics, inspections, measurements, reviews, records, and observations provide suitable coverage for the material acceptance conditions.

  4. 4

    Converged current-state knowledge

    Durable product knowledge has been updated so future participants can understand the accepted system without reconstructing the result from obsolete sources.

  5. 5

    Accountable acceptance

    The relevant human decision owners review the evidence and accept the resulting product state within their authority.

The evidence combination depends on the conditions that applied to the increment. It may include unit and integration tests, AI evaluator metrics, architecture inspection, code-quality review, security and reliability checks, operational observations, performance and cost analysis, and knowledge-consistency review.

Once accepted, the organization should be able to explain where the material conditions came from, how the implementation addressed them, which evidence was used, who had authority to accept the result, and what current-state knowledge now describes the product.

That is the practical value of a trusted increment. The next iteration can begin through the requirement and structured-discussion model in Requirements, Structured Discussion, and Increments, from a product state that the organization has reviewed and chosen to own rather than from a codebase that merely happened to pass its last test run.

Back to top

References

  1. Bjarnason, E., P. Runeson, M. Borg, et al. “Challenges and Practices in Aligning Requirements with Verification and Validation: A Case Study of Six Companies.” Empirical Software Engineering 19, no. 6 (2014): 1809–1855. Springer.
  2. Rhodes, T., F. Boland, E. Fong, and M. Kass. “Software Assurance Using Structured Assurance Case Models.” Journal of Research of the National Institute of Standards and Technology 115, no. 3 (2010): 209–216. NIST.
  3. Liang, P., R. Bommasani, T. Lee, et al. “Holistic Evaluation of Language Models.” Transactions on Machine Learning Research (2023). OpenReview.
  4. Zheng, L., W.-L. Chiang, Y. Sheng, et al. “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena.” Advances in Neural Information Processing Systems 36 (2023). NeurIPS.
  5. Knodel, J., and D. Popescu. “A Comparison of Static Architecture Compliance Checking Approaches.” In Sixth Working IEEE/IFIP Conference on Software Architecture (WICSA 2007), 2007. IEEE.
  6. Tan, W. S., M. Wagner, and C. Treude. “Detecting Outdated Code Element References in Software Repository Documentation.” Empirical Software Engineering 29, article 5 (2024). Springer.
  7. Ricca, F., M. Torchiano, M. Di Penta, M. Ceccato, and P. Tonella. “Using Acceptance Tests as a Support for Clarifying Requirements: A Series of Experiments.” Information and Software Technology 51, no. 2 (2009): 270–283. Elsevier.
  8. Pudlitz, F., F. Brokhausen, and A. Vogelsang. “What Am I Testing and Where? Comparing Testing Procedures Based on Lightweight Requirements Annotations.” Empirical Software Engineering 25 (2020): 2809–2843. Springer.
  9. Inozemtseva, L., and R. Holmes. “Coverage Is Not Strongly Correlated with Test Suite Effectiveness.” In Proceedings of the 36th International Conference on Software Engineering, 435–445, 2014. ACM.

Back to top

Cite this paper

Preparing citation…