Monitor Dashboard and Metrics

Before you begin

Audience: Knowledge architect, governance lead, platform admin

No special role or authentication is required to read the dashboard endpoint beyond standard service access. To interpret the metrics meaningfully, you should understand the Raw to Knowledge registry lifecycle and the AGWMP generation pipeline. For background, see How Raw to Knowledge Builds Governed Knowledge and Flow A vs Flow B.

To query the Prometheus endpoint at /metrics, your monitoring infrastructure (Grafana, Datadog, etc.) must have network access to the Raw to Knowledge service host.

Steps

1. Retrieve the dashboard metrics snapshot

curl -s http://localhost:8000/v1/dashboard/metrics

The response returns a point-in-time snapshot of registry health and pipeline state:

{
  "total_approved": 412,
  "total_disputed": 7,
  "expiring_soon": 23,
  "by_horn_type": {
    "fact": 289,
    "concept": 64,
    "procedure": 38,
    "principle": 21
  },
  "work_items_pending": 14,
  "f2_score": null,
  "precision": null,
  "recall": null,
  "validation_throughput_7d": 0
}

Key fields:

Field	Description
`total_approved`	Count of answers currently in `APPROVED` status
`total_disputed`	Count of answers under active governance review (`DISPUTED`)
`expiring_soon`	Count of approved answers whose `review_due` falls within the next 30 days
`by_horn_type`	Breakdown of approved answers by Horn information type
`work_items_pending`	Count of generation work items in `NEW` or `PENDING_VALIDATION` state
`f2_score`	Recall-weighted F-score (β = 2); null until observability is connected
`precision`	Precision score; null until observability is connected
`recall`	Recall score; null until observability is connected
`validation_throughput_7d`	Reserved field; currently always 0

Note: Dashboard metrics are a point-in-time snapshot queried live from PostgreSQL at the moment you call the endpoint. The Prometheus gauges are also synced at this time. Metrics do not update automatically between calls — if you need a current reading, call the endpoint again.

2. Query the Prometheus metrics endpoint

For continuous monitoring and Grafana dashboards, scrape the Prometheus exposition endpoint:

curl -s http://localhost:8000/metrics

The response is in Prometheus text format, containing all gauges and counters for the Raw to Knowledge service. Configure your Prometheus scrape job to target this endpoint at your desired interval. A typical scrape interval of 60 seconds is appropriate for registry health monitoring.

The metrics data flow from source to display:

flowchart LR
    PG[(PostgreSQL\nRegistry)]
    DS[DashboardService]
    API[GET /v1/dashboard/metrics]
    PROM[Prometheus Gauges]
    SCRAPE[GET /metrics]
    GRAFANA[Grafana / Monitoring]

    PG --> DS
    DS --> API
    DS --> PROM
    PROM --> SCRAPE
    API --> GRAFANA
    SCRAPE --> GRAFANA

3. Interpret the metrics and take action

Use the following guidance to act on what the dashboard shows:

High expiring_soon

If expiring_soon is approaching or exceeding your team's weekly review capacity, prioritize the review backlog before the next readout season. Retrieve the list of expiring answers:

curl -s "http://localhost:8000/v1/registry/answers?status=approved&review_due_before=$(date -v +30d +%Y-%m-%d)"

Assign validators to each answer before its review_due date passes. Answers past their review date remain APPROVED but should be treated as unverified until reviewed.

High total_disputed

A rising total_disputed count means governance decisions are accumulating. Disputed answers are not returned in search or readouts, so unresolved disputes represent a gap in coverage. Pull the disputed list and assign resolution owners:

curl -s "http://localhost:8000/v1/registry/answers?status=disputed"

High work_items_pending

A high work_items_pending count indicates the generation pipeline is backed up. Check worker health:

curl -s "http://localhost:8000/v1/generation/work-items?state=new"
curl -s "http://localhost:8000/v1/generation/work-items?state=pending_validation"

If work items are stuck in NEW for an extended period, the Celery worker may be stopped or unhealthy. Contact your platform administrator to check worker process status.

by_horn_type imbalance

A registry dominated by a single Horn type — most commonly fact — can indicate narrow extraction coverage. If fact accounts for more than 80% of the total and procedure, concept, and principle are sparse, consider reviewing your intake and classification configuration to ensure diverse question types are being captured and classified correctly.

f2_score, precision, and recall are null

These fields require an external observability integration (MLflow or equivalent) and a golden set evaluation run. To produce scores manually:

python tests/golden/evaluate.py

Note: f2_score, precision, and recall will remain null in the dashboard until an observability integration is connected. The golden set evaluation script (tests/golden/evaluate.py) produces these scores when run manually. Results from the script are not automatically written back to the dashboard — they are written to the evaluation output file for offline analysis.

4. Monitor the validation backlog in detail

For a more granular view of what is waiting for human action:

# All new generation work items
curl -s "http://localhost:8000/v1/generation/work-items?state=new"

# All items waiting for validator decision
curl -s "http://localhost:8000/v1/generation/work-items?state=pending_validation"

# Validation queue depth
curl -s "http://localhost:8000/v1/validation/queue/depth"

Use the validation queue depth alongside work_items_pending to assess whether validator capacity is keeping pace with generation output.

Result

You have a clear picture of registry health, governance backlog, and pipeline pressure. You know how many answers are approved, how many are expiring or disputed, and how much work is queued in the generation pipeline. You can connect the /metrics endpoint to Grafana or a compatible monitoring platform for continuous observability without manual polling.