Docker Deployment Architecture

This topic documents the Docker-delivered runtime that is currently defined in docker/docker-compose.yml. It describes the containers that are actually shipped in the repository's Compose stack today, the data paths between them, and the boundaries of that stack.

1. Overview

The current Docker deployment is a compact local or single-host stack with three active containers:

• intake-service runs the FastAPI application and exposes the Raw to Knowledge API.
• postgres is the system-of-record relational store for artifacts, candidates, trend data, validation state, registry objects, and connector sync state.
• redis provides queue and broker infrastructure used by validation queue operations and Celery-based background task coordination.

This Compose file does not currently stand up Neo4j or Weaviate. Those services are part of the broader logical architecture, but they are configured separately and are not delivered by the active Docker stack in this repository.

There is also one optional commented-out service:

• parser-service exists as a future scale-out container pattern, but it is not enabled in the active Compose deployment.

2. Container Diagram

flowchart LR
    U[Client or Operator]
    SRC[Host Source Tree\n../src bind mount]

    subgraph compose["Docker Compose Stack"]
        API[intake-service\nFastAPI + uvicorn]
        PG[(postgres\nPostgreSQL 16)]
        RQ[(redis\nRedis 7 alpine)]
        VOL[(postgres_data\nnamed volume)]
    end

    U -->|HTTP JSON / docs / metrics| API
    API -->|SQL queries and transactions| PG
    API -->|Queue, broker, backend, state ops| RQ
    PG -->|table files, indexes, WAL-backed persistence| VOL
    SRC -->|live application source code| API

3. Container Tables

3.1 Active Containers

Container	Source	Role	Exposed ports	Persistent storage	Key runtime inputs	Notes
intake-service	Built from docker/intake-service/Dockerfile	Runs the FastAPI app, API routes, intake pipeline, connector API, trend extraction, and other in-process services	Host 8001 -> container 8000	Bind mount ../src:/app/src	DATABASE_URL, REDIS_URL, LOG_LEVEL, ENVIRONMENT, CLASSIFICATION_THRESHOLD, BLOCK_VALIDATION_ON_CONFLICT	Depends on healthy postgres and redis; includes NLTK data download at image build time
postgres	postgres:16 image	Primary relational database	Host 5432 -> container 5432	Named volume postgres_data:/var/lib/postgresql/data	POSTGRES_DB, POSTGRES_USER, POSTGRES_PASSWORD	Source of truth for core Raw to Knowledge data; healthchecked with pg_isready
redis	redis:7-alpine image	Queue, broker, backend, and lightweight state store	Host 6379 -> container 6379	None in current Compose file	None required in Compose beyond defaults	Healthchecked with redis-cli ping; used by validation queue and Celery infrastructure

3.2 Optional Container Present in Repo but Not Enabled

Container	Source	Status	Intended role	Exposed ports	Why it is not part of the active stack
parser-service	docker/parser-service/Dockerfile and commented section in docker/docker-compose.yml	Commented out	Future scale-out container for classification-heavy workloads	Host 8002 -> container 8000 if enabled	The current delivered stack keeps classification inside intake-service; this service is only a documented expansion point

3.3 Services Not Delivered by This Compose Stack

Service	In logical architecture	Delivered in current Compose file?	Notes
Neo4j	Yes	No	Required for graph projection and contradiction workflows, but must be provisioned separately
Weaviate	Yes	No	Required for semantic retrieval and parts of generation, but must be provisioned separately
Dedicated Celery worker containers	Yes, operationally	No	The codebase supports them, but this Compose file does not define separate worker containers

4. Edge Tables

4.1 Client or Operator -> intake-service

Field	Detail
Source	External user, script, browser, or ops client
Target	intake-service
Protocol	HTTP
Port path	Host 8001 -> container 8000
Information passed	JSON API requests, health checks, Prometheus scrapes, OpenAPI docs requests, connector pull requests, validation actions, registry queries
Examples	POST /v1/artifacts, POST /v1/connectors/{source_system}/pull, GET /health, GET /metrics
Why it exists	This is the primary ingress path for all application and operations traffic into the stack

4.2 Host Source Tree -> intake-service

Field	Detail
Source	Host repository path ../src mounted into the container
Target	intake-service filesystem at /app/src
Mechanism	Docker bind mount
Information passed	Live Python source files, package modules, local code edits
Why it exists	Supports local development by letting code changes on the host appear immediately inside the running container
Important consequence	The running container uses host-mounted application code rather than only the copied image contents

4.3 intake-service -> postgres

Field	Detail
Source	intake-service
Target	postgres
Protocol	PostgreSQL over TCP via SQLAlchemy asyncpg
Connection string shape	postgresql+asyncpg://raw_to_knowledge:${POSTGRES_PASSWORD}@postgres:5432/raw_to_knowledge
Information passed	SQL statements, transaction boundaries, schema migration effects, persisted artifacts, transcript segments, candidate questions and answers, trend-path rows, validation records, registry rows, connector sync state
Why it exists	PostgreSQL is the primary durable store for the delivered stack

4.4 intake-service -> redis

Field	Detail
Source	intake-service
Target	redis
Protocol	Redis protocol over TCP
Connection string shape	redis://redis:6379/0
Information passed	Validation queue items, Celery broker messages, Celery result backend payloads, lightweight task or queue state operations
Why it exists	Redis provides the asynchronous coordination layer used by queue-backed parts of the application

4.5 postgres -> postgres_data

Field	Detail
Source	postgres container
Target	Named Docker volume postgres_data
Mechanism	Mounted container data directory
Information passed	Table data files, indexes, transaction logs, catalog metadata, durable database state
Why it exists	Preserves database contents across container restarts and recreations

5. Delivery Characteristics

Characteristic	Current state
Compose entrypoint	docker/docker-compose.yml
Application container count	1 active app container
Data service containers	2 active data/infrastructure containers
Worker separation	Not separated in the delivered Compose file
Graph store in stack	No
Vector store in stack	No
Development source mount	Yes
Persistent relational storage	Yes, via postgres_data named volume

6. Operational Interpretation

This deployment is best understood as a developer or compact single-node runtime rather than the full expanded production topology described elsewhere in the architecture documentation. It gives the system:

• a real PostgreSQL database
• a real Redis service
• one API container running the application code

It does not, by itself, deliver:

• a separate worker fleet
• a graph database
• a vector database
• a fully isolated multi-container service-per-subsystem runtime

That makes it appropriate for local development, integration testing, and compact deployments, while still leaving room for the larger target topology documented in the broader architecture set.