States, Statuses, and Queues

Raw to Knowledge uses three distinct state machines — one for candidates moving through the validation workflow, one for published answers in the registry, and one for work items in the generation pipeline. This reference defines every state, who sets it, and what transitions are permitted.

Validation states

These states apply to CandidateAnswer records as they move through the human validation workflow.

State	Meaning	Who sets it	Next permitted states
`PENDING`	Candidate has been extracted and classified. Awaiting human attention.	System (extraction)	`IN_REVIEW`
`IN_REVIEW`	A validator has claimed the candidate and is actively reviewing it.	Validator (claim action)	`APPROVED`, `REVISED`, `REJECTED`, `ESCALATED`
`APPROVED`	Candidate was accepted as-is. Ready to be published to the registry.	Validator	`SUPERSEDED` (after registry publish of a newer version)
`REVISED`	Candidate was accepted with edited text. The `ValidationRecord.revised_text` field holds the approved version.	Validator	`SUPERSEDED`
`REJECTED`	Candidate was rejected. It will not be published.	Validator	(terminal — no further transitions)
`ESCALATED`	Candidate requires expert input. A `WorkItem` is created for generation or specialist review.	Validator	`PENDING` (if reworked and re-submitted), `CLOSED`
`SUPERSEDED`	A newer version of this answer has been published. This candidate is no longer the current version.	System (registry supersede action)	(terminal)
`DISPUTED`	The answer is under dispute — conflicting information has been identified. Review required.	System (conflict detection) or validator	`IN_REVIEW`, `SUPERSEDED`, `RETIRED`
`RETIRED`	The answer has reached end-of-life and is no longer operationally active.	Validator or administrator	(terminal)

State machine rules: - A validator cannot claim a candidate that is not in PENDING. - A candidate cannot be approved or revised without first being in IN_REVIEW. - Attempting an illegal transition returns 422 ValidationStateError. - OPEN_ISSUE Horn type candidates cannot be approved — they must be escalated. Once the generation pipeline produces a validated answer, it will be a new candidate of a concrete type.

Answer statuses

These statuses apply to ApprovedAnswer records in the registry after publication.

Status	Meaning	How reached	Effect on retrieval and readout
`APPROVED`	The answer is current and authoritative.	Registry publish from a validated `CandidateAnswer`.	Included in semantic search and readout. Returned by default.
`SUPERSEDED`	This answer has been replaced by a newer version. The `supersedes_id` chain links to the replacement.	Registry supersede action (POST `/v1/registry/answers/{id}/supersede`).	Excluded from default retrieval. Accessible by explicit ID for audit purposes.
`DISPUTED`	A conflict has been identified. The answer should not be relied on until resolved.	Conflict detection creates a `ConflictFlag`, which can be escalated to `DISPUTED` status. Or directly via dispute action.	Excluded from default retrieval. Flagged in readout results.
`RETIRED`	The answer has been intentionally retired. It is no longer current or applicable.	Retire action (POST `/v1/registry/answers/{id}/retire`).	Excluded from all retrieval and readout. Retained in the database for audit.

Immutability note: No status transition physically modifies or removes an ApprovedAnswer. The record is preserved in all cases. Status transitions are the only lifecycle operations the API exposes. There is no delete operation.

Work item states

These states apply to WorkItem records as they move through the generation pipeline.

State	Meaning	Who sets it	Next permitted states
`NEW`	Work item created. Awaiting generation trigger.	System (escalation) or user (manual creation)	`GENERATING`, `CLOSED`
`GENERATING`	A Celery task is actively running LLM generation for this work item.	System (generation task start)	`PENDING_VALIDATION`, `ESCALATED`
`PENDING_VALIDATION`	A `GeneratedCandidate` has been produced. Awaiting human validation review.	System (generation task completion)	`IN_CLASSIFICATION`, `READY_FOR_REVIEW`
`IN_CLASSIFICATION`	The generated candidate is being classified before review assignment.	System (classification run)	`READY_FOR_REVIEW`
`READY_FOR_REVIEW`	Classified and queued for human review.	System	`VALIDATED`, `NEEDS_REWORK`, `ESCALATED`
`VALIDATED`	The generated answer has been validated and published to the registry.	System (registry publish)	`CLOSED`
`NEEDS_REWORK`	Reviewer rejected the generated answer. Work item returned for re-generation.	Reviewer	`GENERATING`, `CLOSED`
`CLOSED`	Work item is complete. No further action required.	System or administrator	(terminal)
`ESCALATED`	Generation or validation could not complete. Human expert intervention required.	System (LLM failure, `no_content`) or reviewer	`CLOSED`

Common escalation reasons: - no_content — fewer than 2 RAG sources were found to ground the generation. - LLM error: <detail> — the LLM API returned an error. - LLM circuit open — the circuit breaker has tripped after repeated LLM failures.

Queue behavior

Raw to Knowledge uses two types of queues with different semantics.

Redis validation queue

The validation queue holds candidates awaiting human review. It is implemented as a Redis list, partitioned by Horn type.

Property	Value
Key pattern	`raw_to_knowledge:val_queue:{horn_type}`
Ordering	FIFO within each Horn type partition
Dequeue operation	`GET /v1/validation/queue?horn_type={type}`
Population	Candidates enter the queue when classification completes and `validation_state = PENDING`
Partition reason	Different Horn types are routed to validators with the appropriate KGS authority. A FACT validator is not shown PROCEDURE candidates.

Queue isolation example: If three validators are working simultaneously, a validator requesting ?horn_type=FACT will only dequeue FACT candidates. A PROCEDURE candidate will not appear in their queue even if it has been waiting longer.

Celery task queues

Celery workers consume from named queues. Each queue type has a recommended concurrency level based on task characteristics.

Queue	Tasks	Recommended concurrency	Notes
`raw_to_knowledge_generation`	`generate_answer_task`	4	I/O-bound (LLM API calls). Higher concurrency is safe. Subject to circuit breaker.
`raw_to_knowledge_graph`	`sync_graph_task`, `detect_contradictions_task`	2	Mix of PostgreSQL reads and Neo4j writes. Lower concurrency avoids Neo4j write contention.

Tasks are enqueued by the API service (202 Accepted response) and dequeued by Celery workers. Workers must be running for tasks to execute. If no workers are consuming a queue, tasks accumulate in Redis and execute when workers come online.

Monitoring queue depth:

# See how many tasks are waiting in each queue
celery -A raw_to_knowledge.infrastructure.celery_app inspect reserved

# See active tasks across all workers
celery -A raw_to_knowledge.infrastructure.celery_app inspect active