Skip to content

Review Extracted Candidates

Before you begin

Roles required: Validator, support engineer.

You need the artifact_id from a completed ingest, or you can review the full candidate queue without filtering by artifact. Segmentation must have completed — allow a few seconds after ingest before retrieving candidates.

Assumed state: At least one artifact has been ingested. See Ingest a Conversation Source.

Steps

1. Retrieve the candidate list

Send a GET request to /v1/candidates. You can optionally filter by artifact_id to focus on a single ingest batch.

# All candidates (paginated)
curl -s "http://localhost:8000/v1/candidates"

# Candidates from a specific artifact
curl -s "http://localhost:8000/v1/candidates?artifact_id=art_01j2k3l4m5n6p7q8r9s0"

The response is a paginated list. Each candidate object includes the following key fields:

Field Description
candidate_id Unique identifier — use this in all subsequent API calls
question_text The question or knowledge gap detected in the source
flow_type FLOW_A (paired answer found) or FLOW_B (no answer; needs generation)
confidence Extraction confidence score, 0.0–1.0
privacy_class Privacy classification applied by content analysis
artifact_id The source artifact this candidate was extracted from
created_at Timestamp of extraction

Review the list to understand the volume, distribution of flow types, and any candidates flagged with low confidence or elevated privacy classification.

Note: Extraction confidence measures the quality of question-answer detection from the raw source. It is distinct from classification confidence, which is about Horn type assignment and is computed separately during the classify step.

2. Retrieve the paired answer for Flow A candidates

For candidates with flow_type: "FLOW_A", an answer text was detected in the source alongside the question. Retrieve it before proceeding to classification:

curl -s "http://localhost:8000/v1/candidates/cand_01j2k3l4m5n6p7q8r9t1/answers"

The response includes the answer_text as extracted from the source. Review it for completeness and accuracy. If the answer text is clearly wrong or incomplete, note it — you will have the opportunity to correct it during validation using the REVISED decision.

3. Evaluate extraction confidence

The confidence field on each candidate indicates how reliably the segmenter detected the question-answer boundary.

Score range Interpretation Recommended action
0.9 and above High confidence Proceed to classification without additional review
0.75 – 0.89 Moderate confidence Spot-check the question and answer text before classifying
Below 0.75 Low confidence Review carefully; the extracted text may be incomplete or misaligned

Low-confidence candidates are not automatically removed. They follow the same pipeline. The confidence score is a signal for where human attention is most needed.

Note: Candidates are read-only. You cannot edit the question_text or extracted answer_text at this stage. If the extraction is incorrect, proceed through classification and validation — use the REVISED decision at the validation step to supply corrected text.

4. Review privacy classification

Each candidate carries a privacy_class assigned during content analysis. Common values include PUBLIC, INTERNAL, and PII_CONTAINING.

Any candidate with a privacy_class of PII_CONTAINING must not be published to the registry without explicit review. Before classifying or validating these candidates:

  • Confirm the content is within the declared legal basis for the source artifact
  • Confirm your governance lead has approved publication of this content
  • Consider whether the answer text should be generalized or redacted before entering the registry

Candidates flagged as PII_CONTAINING can still move through the full pipeline — the flag does not block processing. It is a governance signal that requires human judgment.

5. Note Flow B candidates for generation

Candidates with flow_type: "FLOW_B" have a detected question but no paired answer in the source material. These candidates require a generation work item to produce a draft answer before classification can be meaningful.

Make a note of the candidate_id values for all FLOW_B candidates. Coordinate with your team to:

  1. Submit a generation work item via your content or LLM tooling
  2. Once a draft answer exists, proceed to classification with the draft text

FLOW_B candidates are visible in the candidate list but will not have answer text available at /v1/candidates/{id}/answers.

Result

You have reviewed the candidate list and identified which candidates are ready for classification, which need a generation work item, and which require closer attention due to low extraction confidence or elevated privacy classification.

See also: Classify a Candidate Answer; Ingest a Conversation Source; Horn Information Types