Milestone 3 — Human Validation Workflow

Status: Complete
Completed: 2026-06-07

Goal

Ensure that no candidate answer reaches the knowledge graph without a human validation decision from an authorised reviewer.

Stakeholder story: "Every answer that enters the knowledge graph has been explicitly approved by someone with the authority to approve answers of that type. Conflicts between candidates are flagged automatically."

What Was Built

State Machine

ValidationStateMachine enforces a nine-state transition table derived directly from ValidationState.permitted_transitions(). The machine is a pure domain service — it operates only on values and returns new immutable ValidationAssignment instances.

States:

State	Description
pending	Queued, awaiting a validator
in_review	Claimed by a validator
approved	Approved by an authorised reviewer
revised	Revised answer text submitted; triggers re-classification
rejected	Definitively rejected (terminal)
escalated	Routed to a higher authority
superseded	Replaced by a newer decision (terminal)
disputed	Flagged for adjudication post-approval
retired	Removed from active use (terminal)

Transitions:

• pending → in_review (claim)
• in_review → approved | revised | rejected | escalated
• revised → approved | in_review
• escalated → in_review
• approved → superseded | disputed | retired
• disputed → in_review
• rejected | superseded | retired → none (terminal)

Role-Based Authority Table

AuthorityChecker enforces the KGS authority table mapping Horn types to approved reviewer roles. Authority is checked at the moment of an APPROVED decision — not at assignment time.

Horn Type	Required Roles
fact	subject_matter_expert, principal
concept	content_strategist, principal
procedure	technical_writer, principal
process	engineer, principal
principle	principal only
reference	content_strategist, principal
troubleshooting	support_engineer, principal
recommendation	consultant, principal
open_issue	(empty — never approvable)

Grammar-Level Conflict Detection

ConflictDetector compares new APPROVED assignments against all active assignments. A conflict exists when two candidates assert different object_value values for the same subject + predicate + scope triple. Conflicts are recorded as ConflictFlag objects and attached to the assignment. When block_on_conflict=True (env: BLOCK_VALIDATION_ON_CONFLICT), APPROVED decisions with conflicts raise ConflictDetectedError before persisting.

Validation Queue (Redis + In-Memory)

RedisValidationQueue routes assignments to per-Horn-type Redis lists (raw_to_knowledge:val_queue:{horn_type}) using RPUSH/LPOP (FIFO). InMemoryValidationQueue is used for tests and development. The queue implementation is swapped via dependency injection without changing the application layer.

Immutable Audit Trail

ValidationRecord is a frozen Pydantic v2 model. Once created it is never modified — each decision produces a new record with prior_record_id linking to the previous one. The repository offers only save_record, never update_record.

Registry Write Guard

ValidationService.require_approved_record(candidate_id) raises UnvalidatedRegistryWriteError when called for a candidate with no APPROVED validation record. M4 (Knowledge Graph Write) calls this as its first action.

API

All endpoints are under /v1/validation/:

Method	Path	Description
POST	/assignments	Assign a candidate for validation
GET	/assignments	List assignments (filterable by state, horn_type, actor)
GET	/assignments/{id}	Get assignment by ID
POST	/assignments/{id}/claim	Claim (PENDING → IN_REVIEW)
POST	/assignments/{id}/decide	Submit a decision
GET	/candidates/{id}/history	Full audit trail for a candidate
GET	/candidates/{id}/state	Current state for a candidate
GET	/queue/depth	Pending counts per horn type

HTTP status codes:

• 201 — Created (assignment or decision)
• 403 — Role not authorised for approval
• 404 — Assignment not found
• 409 — Invalid state transition or conflict blocked
• 422 — Invalid enum value or missing required field

Database Schema (Migration 003)

Three new tables:

• validation_assignments — mutable work items (state, actor, record/flag ID lists)
• validation_records — immutable decision audit trail
• conflict_flags — grammar conflicts detected between assignments

Configuration

Env var	Default	Description
REDIS_URL	redis://localhost:6379/0	Redis connection URL
BLOCK_VALIDATION_ON_CONFLICT	false	When true, conflicts block APPROVED decisions

Tests

Unit tests (no DB required): - tests/unit/test_state_machine.py — exhaustive valid and invalid transition matrix, claim, flag_conflict, terminal/approved properties - tests/unit/test_authority_checker.py — all Horn types, authorised/unauthorised roles, OPEN_ISSUE enforcement, custom table override - tests/unit/test_conflict_detector.py — GrammarSnapshot normalisation, same_key, conflicts_with, multi-conflict detection, self-comparison skip

Integration tests (requires DATABASE_URL): - tests/integration/test_validation_roundtrip.py — full assign→claim→decide pipeline, authority enforcement, REVISED triggers reclassification log, conflict detection, immutability guard, registry write guard

Design Decisions

• Frozen models for state: ValidationAssignment uses Pydantic frozen + model_copy(update=...) to produce new instances on each transition. This keeps the domain free of mutability side-effects.
• Redis FIFO per Horn type: Each Horn type gets its own list key so validators can subscribe only to types they're authorised to review — prevents routing PRINCIPLE assignments to technical writers.
• Conflict detection on APPROVED only: Running detection at REVISED or REJECTED would produce noise. Running at APPROVED means only answers that would enter the knowledge graph are checked.
• Log event for re-classification: In M3, REVISED decisions emit a structured log event. M6 will hook a Celery task to this log line; no plumbing change required.