Architecture
coreX Platform is a full HAProxy + Coraza WAF control plane. It separates the control plane (FastAPI backend + React UI that manage configuration) from the data plane (HAProxy + Coraza SPOA that process traffic). This page walks through the architecture from several angles.
Control Plane vs Data Plane
┌─────────────────────────────────────────────────────────────────┐
│ CONTROL PLANE │
│ (manages config, never touches user traffic) │
│ │
│ ┌─────────────┐ ┌──────────────┐ ┌──────────────────────┐ │
│ │ frontend │ │ api │ │ postgres │ │
│ │ React UI │──▶│ FastAPI │──▶│ config + state DB │ │
│ │ :3000/3443 │ │ :8000 │ │ :5432 │ │
│ └─────────────┘ └──────┬───────┘ └──────────────────────┘ │
│ │ │
│ ▼ │
│ ┌────────────┐ │
│ │ valkey │ cache · rate limits · │
│ │ :6379 │ token blocklist · task queue │
│ └────────────┘ │
└───────────────────────────┬─────────────────────────────────────┘
│ writes haproxy.cfg + coraza.cfg
│ via shared volume + stats socket
▼
┌─────────────────────────────────────────────────────────────────┐
│ DATA PLANE │
│ (processes user traffic, never managed by users directly) │
│ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ corex (HAProxy) │ │
│ │ :80 HTTP :443 HTTPS/HTTP3 :8404 stats :5555 DP │ │
│ │ │ │
│ │ ┌──────────────┐ SPOE ┌──────────────────────┐ │ │
│ │ │ listeners │───filter──▶│ coraza-spoa (WAF) │ │ │
│ │ │ backends │◀──verdict──│ :9000 │ │ │
│ │ └──────┬───────┘ └──────────────────────┘ │ │
│ │ │ routes to your origin servers │ │
│ └──────────┼───────────────────────────────────────────────┘ │
└─────────────┼───────────────────────────────────────────────────┘
▼
your backendsThe control plane generates haproxy.cfg and coraza.cfg on the shared haproxy-data volume, validates them with haproxy -c, then reloads HAProxy via the stats socket (or Data Plane API in HA mode). The data plane reads the config and serves traffic — it has no knowledge of the database or UI.
Service Topology
Every service in the stack and how it connects to the others:
┌────────────┐
│ browser │
└─────┬──────┘
│
┌─────────────┼─────────────┐
▼ ▼ ▼
:3000 HTTP :3443 HTTPS :80/:443 ──▶ origins
│ │ │
┌──────┴─────┐ ┌────┴─────┐ ┌────┴───────────────┐
│ frontend │ │ frontend │ │ corex │
│ (nginx) │ │ (nginx) │ │ (HAProxy) │
└──────┬─────┘ └────┬─────┘ │ ┌──────────────┐ │
│ │ │ │ coraza-spoa │ │ SPOE
│ /api/v1/* │ │ │ :9000 │ │ filter
└─────────────┼───────▶│ └──────────────┘ │
│ └────┬───────────────┘
│ │ stats socket
▼ ▼
┌───────────────────────────┐
│ api │ FastAPI
│ (config gen, samplers, │
│ task queue, auth) │
└─────┬──────────┬──────────┘
│ │
┌──────────┘ └──────────┐
▼ ▼
┌──────────┐ ┌──────────────┐
│ postgres │ │ valkey │
│ :5432 │ │ :6379 │
└──────────┘ └──────┬───────┘
│ shared
┌───────────────┼──────────────┐
▼ ▼ ▼
┌──────────┐ ┌────────────┐ ┌───────────┐
│ cap │ │ varnish │ │ vector │
│ :3001 │ │ (disk │ │ (log │
│ CAPTCHA │ │ cache) │ │ pipeline) │
└──────────┘ └────────────┘ └───────────┘
Optional MCP layer (when MCP_GATEWAY_ENABLED=true):
┌───────────────────┐ ┌──────────────────┐
│ mcp-gateway-rs │ register │ mcp-server │
│ :8089/mcp │◀───────── │ :8082/mcp │
│ (policy/DLP/ │ │ (control plane │
│ guardrails) │ │ as MCP tools) │
└───────────────────┘ └──────────────────┘| Service | Role | Always on? |
|---|---|---|
frontend | React UI served by nginx | Yes |
api | FastAPI control plane (config generation, auth, samplers, task queue) | Yes |
corex | HAProxy data plane (traffic routing, TLS, WAF filter, caching) | Yes |
coraza-spoa | Coraza WAF SPOA (OWASP CRS, custom rules) | Yes (when CORAZA_SPOA_ENABLED=true) |
valkey | Redis-compatible store (cache, rate limits, token blocklist, task queue) | Yes |
postgres | Config + state database | Yes |
cap | Cap CAPTCHA challenge service | When CAPTCHA enabled |
varnish | Varnish disk cache sidecar | When DISK_CACHE_ENABLED=true |
vector | Vector log pipeline (sources + sinks) | When log sinks configured |
mcp-gateway-rs | Rust MCP gateway proxy (policy/DLP/guardrails) | When MCP_GATEWAY_ENABLED=true |
mcp-server | coreX MCP server (control plane as MCP tools) | When MCP_GATEWAY_ENABLED=true |
Request Flow
How a single HTTP request traverses the data plane:
client
│
▼ :443
┌──────────────────────────────────────────────────────────────────┐
│ corex (HAProxy) — frontend listener │
│ │
│ 1. TLS termination (cert from CERT_DIR) │
│ 2. JA4 fingerprint capture (lua/ja4.lua → txn.ja4) │
│ 3. Request fingerprint (lua/req_fp.lua → txn.req_fp) │
│ 4. GeoIP/ASN enrichment (geoip2 fetch → txn.country/asn) │
│ 5. Security rules (first-match-wins, ordered) │
│ ├─ allow / deny / log / tarpit / challenge / captcha │
│ 6. Rate limiting (stick-table sc-inc-gpc0, sliding window) │
│ 7. WAF filter (SPOE → coraza-spoa) │
│ ├─ OWASP CRS anomaly scoring │
│ ├─ custom SecRules │
│ └─ verdict: allow / block / log / challenge │
│ 8. API Armor (lua/rust module, when enabled) │
│ ├─ JSON schema validation │
│ ├─ API-key / JWT auth │
│ ├─ GraphQL depth/complexity analysis │
│ └─ behavioral profiling + anomaly detection │
│ 9. Page Protect (CSP enforcement, script inventory) │
│ 10. Caching (memory cache + optional Varnish disk cache) │
│ 11. Compression (Brotli / Zstd / Gzip, per-backend) │
│ 12. Response transforms (replace / inject / mask, per-backend) │
│ 13. Image conversion (WebP, per-backend) │
│ 14. Backend selection (roundrobin / leastconn / static / ...) │
└──────────────────────────┬───────────────────────────────────────┘
│
▼
origin server
│
▼ response
┌──────────────────────────────────────────────────────────────────┐
│ corex (HAProxy) — response path (reverse order) │
│ • response transforms apply │
│ • image conversion applies │
│ • compression applies │
│ • caching stores (if cacheable) │
│ • structured JSON log line emitted (stdout + Vector ring) │
└──────────────────────────────────────────────────────────────────┘Each numbered stage is configurable per-listener or per-backend from the UI. The order is fixed by the HAProxy config generation pipeline — security checks run before routing, transforms run on the response.
Config Generation Pipeline
When you click Apply Config, the control plane runs a multi-step pipeline. Every step writes to the shared haproxy-data volume so the data plane picks it up on reload.
Apply Config (UI)
│
▼
┌─────────────────────────────────────────────────────────────┐
│ 1. generate_config() │
│ haproxy.py — builds haproxy.cfg from DB rows: │
│ global · defaults · stats frontend · data plane API │
│ listeners · backends · servers · peers (HA) │
│ security rules · rate limits · WAF SPOE filter │
│ caching · compression · response transforms │
│ CAPTCHA proxy · MCP gateway backend · MCP upstreams │
└──────────────────────────┬──────────────────────────────────┘
│
┌───────────────────┼───────────────────┐
▼ ▼ ▼
┌─────────────┐ ┌──────────────┐ ┌──────────────────┐
│ 2. security │ │ 3. MCP config│ │ 4. resp transform│
│ list files │ │ bundle │ │ files │
│ (.lst) │ │ │ │ │
└─────────────┘ └──────────────┘ └──────────────────┘
┌───────────────────┼───────────────────┐
▼ ▼ ▼
┌────────────┐ ┌──────────────┐ ┌──────────────────┐
│ risk rules │ │ API Armor │ │ Page Protect │
│ data file │ │ files │ │ beacon JS │
└────────────┘ └──────────────┘ └──────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ 5. Coraza config (when CORAZA_SPOA_ENABLED) │
│ coraza.cfg (SPOE) + coraza-spoa.yaml (directives) │
└──────────────────────────┬──────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ 6. validate — haproxy -c -f haproxy.cfg │
│ (runs in the corex container via RuntimeBackend) │
│ ✗ invalid → raise, abort apply, no reload │
└──────────────────────────┬──────────────────────────────────┘
│ ✓ valid
▼
┌─────────────────────────────────────────────────────────────┐
│ 7. write files to disk (haproxy-data volume) │
│ + snapshot the config into the DB (for revert) │
└──────────────────────────┬──────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ 8. reload HAProxy │
│ single instance: stats socket (reload) │
│ HA mode: Data Plane API push to each instance │
│ + restart coraza-spoa if config changed │
│ + reload vector pipeline if vector.toml changed │
└─────────────────────────────────────────────────────────────┘The validation step (6) is the safety net — a generated config that fails haproxy -c never reaches disk, so the running data plane is never broken by a bad apply.
Runtime Backend Abstraction
The control plane needs to run commands inside sibling containers (haproxy -c, coraza-spoa restart, varnish VCL reload, vector exec). The RuntimeBackend interface abstracts this so the same backend code works on Docker Compose and Kubernetes.
backend/app/services/runtime/
│
┌─────────┴─────────┐
│ get_runtime() │ COREX_RUNTIME=auto
│ (singleton) │ detects at startup
└─────────┬─────────┘
│
┌──────────────────┼──────────────────┐
▼ ▼ ▼
┌───────────────┐ ┌──────────────┐ ┌──────────────┐
│ DockerRuntime │ │ Kubernetes │ │ NullRuntime │
│ │ │ Runtime │ │ │
│ docker SDK │ │ k8s API │ │ no-op: all │
│ exec_run │ │ exec into │ │ ops return │
│ container │ │ own pod via │ │ unavailable │
│ logs/restart │ │ downward API │ │ │
└───────────────┘ └──────────────┘ └──────────────┘
│ │
/var/run/docker.sock KUBERNETES_SERVICE_HOST
exists + SA token exists| Operation | Docker | Kubernetes |
|---|---|---|
haproxy -c validate | docker exec corex haproxy -c -f ... | kubectl exec <pod> -c corex -- haproxy -c -f ... |
| HAProxy logs | container.logs(tail=N) | kubectl logs <pod> -c corex --tail=N |
| Restart Coraza | docker restart coraza-spoa | Signal PID 1 in the coraza container (shared PID namespace) |
| Varnish VCL reload | docker exec varnish varnishadm ... | kubectl exec <pod> -c varnish -- varnishadm ... |
| Vector exec | docker exec vector vector ... | kubectl exec <pod> -c vector -- vector ... |
The shareProcessNamespace: true setting on the K8s sidecar pod is what makes single-container restarts possible — the API container can signal Coraza’s PID 1 directly without rolling the whole pod.
High Availability Topologies
coreX Platform supports three HA topologies. The control plane config is identical across all three; only the deployment mechanism and failover strategy differ.
Single instance (default)
┌─────────────────────────────────────┐
│ one host │
│ │
│ corex ── coraza-spoa │
│ │ │
│ ▼ │
│ api ── valkey ── postgres │
│ │ │
│ frontend │
└─────────────────────────────────────┘No failover. Use for dev, staging, or non-critical production. HA_ENABLED=false.
Docker Compose HA (keepalived VRRP)
host A host B
┌────────────────────┐ ┌─────────────────────┐
│ corex (MASTER) │ VRRP VIP │ corex2 (BACKUP) │
│ keepalived │◀────────────▶│ keepalived │
│ coraza-spoa │ stick-table │ coraza-spoa2 │
│ │ peers :10000│ │
│ api │ │ (no api — config │
│ valkey (primary) │ sentinel │ pushed via DP API)│
│ valkey-replica │◀────────────▶│ valkey-replica │
│ valkey-sentinel │ quorum │ valkey-sentinel │
└────────────────────┘ └─────────────────────┘
│
▼ KEEPALIVED_VIP (floating IP)
clientsHA_ENABLED=true + docker-compose.ha.yml. Keepalived provides VRRP failover for the VIP; Valkey Sentinel handles primary/replica failover; stick-tables replicate over the peer port (10000) so rate-limit and connection state survives failover. The control plane (api) runs on the primary host and pushes config to both HAProxy instances via the Data Plane API.
Docker Swarm HA (ingress mesh)
┌──────────────────────────────────────────────────────────┐
│ Swarm cluster │
│ │
│ ┌─────────────┐ ingress mesh ┌─────────────┐ │
│ │ corex (r1) │◀─── VIP ──────▶│ corex (r2) │ ← no │
│ │ coraza-spoa │ routing │ coraza-spoa │ keep- │
│ │ │ │ │ alived │
│ └──────┬──────┘ └──────┬──────┘ │
│ │ tasks.corex DNS (dnsrr) │ │
│ └──────────── peers ───────────┘ │
│ │
│ valkey (primary) ── valkey-replica ── 3× sentinel │
│ api (1 replica, pushes config to all HAProxy tasks) │
└──────────────────────────────────────────────────────────┘HA_ENABLED=true + SWARM_MODE=true. Swarm’s ingress mesh provides the VIP and automatic failover — keepalived is skipped. Stick-table peers discover each other via tasks.corex DNS with endpoint_mode: dnsrr.
Kubernetes (sidecar pod)
┌───────────────────────────────────────────────────────┐
│ Kubernetes cluster │
│ │
│ ┌──────────────────────────────────────────────────┐ │
│ │ sidecar pod (StatefulSet, shareProcessNamespace)│ │
│ │ │ │
│ │ ┌─────────┐ shared ┌────────────┐ │ │
│ │ │ api │ volume │ corex │ │ │
│ │ │ │ + unix │ (HAProxy) │ │ │
│ │ │ │ socket │ │ │ │ │
│ │ └─────────┘ │ ┌──┴─────┐│ │ │
│ │ │ │coraza ││ │ │
│ │ │ │ spoa ││ │ │
│ │ │ └────────┘│ │ │
│ │ │ ┌───────┐ │ │ │
│ │ │ │varnish│ │ optional │ │
│ │ │ └───────┘ │ │ │
│ │ └────────────┘ │ │
│ └──────────────────────────────────────────────────┘ │
│ │
│ postgres (StatefulSet or external) │
│ valkey (StatefulSet or external) │
│ frontend (Deployment + Service) │
└───────────────────────────────────────────────────────┘The tightly-coupled services share a single pod because they need a shared volume, a Unix socket, and localhost networking. COREX_RUNTIME=kubernetes (auto-detected from the K8s service account token) switches the backend to the Kubernetes API for exec/logs/restart.
Security Layers
The data plane applies multiple security layers in a fixed order. Each layer is independently configurable per-listener or per-backend.
incoming request
│
▼
┌───────────────────────────────────────────────────────────────┐
│ Layer 1: Enrichment (no action taken) │
│ • JA4 TLS fingerprint (lua/ja4.lua) │
│ • HTTP request fingerprint (lua/req_fp.lua) │
│ • GeoIP country + ASN (geoip2 fetch) │
│ → stored in txn.* vars, available to all later layers │
└──────────────────────────┬────────────────────────────────────┘
▼
┌───────────────────────────────────────────────────────────────┐
│ Layer 2: Security Rules (first-match-wins, ordered) │
│ • coreX expression language (field op value, AND/OR/NOT) │
│ • matches against JA4, GeoIP, ASN, IP/CIDR, path, header │
│ • actions: allow · deny · log · tarpit · challenge · captcha│
│ • consumes Security Lists (network/ASN/GeoIP/JA4 .lst files)│
└──────────────────────────┬────────────────────────────────────┘
▼
┌───────────────────────────────────────────────────────────────┐
│ Layer 3: Rate Limiting (stick-table based) │
│ • fixed-window and sliding-window counters │
│ • per-endpoint scoping (path, header, IP) │
│ • block durations + tarpit │
│ • actions: deny · tarpit · challenge · captcha · log │
└──────────────────────────┬────────────────────────────────────┘
▼
┌───────────────────────────────────────────────────────────────┐
│ Layer 4: WAF (Coraza SPOA via SPOE filter) │
│ • OWASP CRS anomaly scoring │
│ • custom SecRules │
│ • remote rule sets (downloaded + SHA256 verified) │
│ • conditional exceptions (per-path, per-rule) │
│ • WAF rate limiting (sliding window inside Coraza) │
│ • verdict: allow · block · log · challenge │
└──────────────────────────┬────────────────────────────────────┘
▼
┌───────────────────────────────────────────────────────────────┐
│ Layer 5: API Armor (Rust Lua module, when enabled) │
│ • OpenAPI/JSON schema validation → 400 on violation │
│ • API-key auth → 401 on failure │
│ • JWT auth → 401 on failure │
│ • GraphQL depth/complexity/field analysis │
│ • behavioral profiling + anomaly detection → 403 │
└──────────────────────────┬────────────────────────────────────┘
▼
┌──────────────────────────────────────────────────────────────┐
│ Layer 6: Page Protect (client-side, CSP enforcement) │
│ • Content-Security-Policy headers │
│ • script inventory + code-change detection │
│ • violation report collection (beacon endpoint) │
└──────────────────────────┬───────────────────────────────────┘
▼
┌──────────────────────────────────────────────────────────────┐
│ Layer 7: CAPTCHA challenge (action from layers 2-5) │
│ • Native (Cap) · reCAPTCHA · Cloudflare Turnstile │
│ • challenge page served at /_cap/challenge │
│ • verified tokens unblock the original request │
└──────────────────────────┬───────────────────────────────────┘
▼
route to backend| Layer | Configurable from | Scope | Feature docs |
|---|---|---|---|
| Security Rules | Security > Security Rules | per-listener | Security Rules |
| Rate Limiting | Traffic > Rate Limiting | per-listener / per-path | Rate Limiting |
| WAF | Security > WAF | per-listener | WAF |
| API Armor | Security > API Armor | per-listener | API Armor |
| Page Protect | Security > Page Protect | per-backend | Page Protect |
| CAPTCHA | Security > CAPTCHA | global (provider) + per-rule (action) | CAPTCHA |
Observability
Three independent observability pipelines feed the UI dashboards and external sinks.
┌───────────────────────────────────────────────────────────────┐
│ HAProxy metrics │
│ stats socket (show info / show stat) │
│ │ every METRICS_SAMPLE_INTERVAL_SECONDS (30s) │
│ ▼ │
│ metrics.py sampler → MetricSnapshot table → /metrics API │
│ │ → frontend Metrics.tsx │
│ ▼ │
│ pruned after METRICS_RETENTION_DAYS (7d) │
└───────────────────────────────────────────────────────────────┘
┌───────────────────────────────────────────────────────────────┐
│ WAF metrics │
│ coraza-spoa.log (JSON events) │
│ │ every WAF_METRICS_SAMPLE_INTERVAL_SECONDS (10s) │
│ ▼ │
│ waf_metrics.py sampler (tails log, parses events) │
│ │ → WafMetric table → /waf/metrics API │
│ │ → frontend Metrics.tsx (WAF panel) │
│ ▼ │
│ raw log pruned to WAF_LOG_RETENTION_LINES (500) │
│ DB events pruned after WAF_METRICS_RETENTION_DAYS (7d) │
└───────────────────────────────────────────────────────────────┘
┌───────────────────────────────────────────────────────────────┐
│ Request logs → Vector pipeline → external sinks │
│ HAProxy emits structured JSON log line │
│ │ │
│ ├──▶ stdout (docker compose logs corex) │
│ │ │
│ └──▶ log ring@vector_tcp → tcp://vector:601 │
│ │ │
│ ▼ │
│ vector container (reads vector.toml) │
│ sources: corex (tcp) · waf (file) · mcp (file) │
│ │ │
│ ▼ VRL transforms (parse, enrich, tag) │
│ sinks (admin-configured, Fernet-encrypted secrets): │
│ aws_s3 · azure_logs · datadog · elasticsearch │
│ http · new_relic · splunk_hec │
└───────────────────────────────────────────────────────────────┘The Vector pipeline is managed entirely from the UI (Logging > Log Pipeline). Sink credentials are encrypted in the database with VECTOR_SECRETS_KEY (falling back to SECRET_KEY) and never appear in API responses, previews, or diffs without explicit secret inclusion.
MCP Gateway Architecture
When MCP_GATEWAY_ENABLED=true, the stack adds an MCP (Model Context Protocol) layer so AI agents can interact with the control plane.
AI agent (Devin CLI, Claude Code, Cursor, Windsurf, ...)
│
│ MCP over Streamable HTTP
▼
┌──────────────────────────────────────────────────────────────┐
│ mcp-gateway-rs :8089/mcp │
│ (Rust — policy / DLP / guardrails proxy) │
│ │
│ • per-server auth (bearer, encrypted at rest) │
│ • policy enforcement (allow/deny per tool) │
│ • DLP scanning (redact secrets in responses) │
│ • guardrails (rate limits, audit logging) │
│ • config bundle from /app/data/mcp/config.bundle.json │
└──────────────────────────┬───────────────────────────────────┘
│ proxies tool calls
│ (tools appear as corex-manager__*)
▼
┌──────────────────────────────────────────────────────────────┐
│ mcp-server :8082/mcp │
│ (Python — exposes the control plane as MCP tools) │
│ │
│ • auto-discovers 339 tools from FastAPI routes │
│ • executes in-process via ASGI transport (no HTTP hop) │
│ • service admin JWT (cached 20min) │
│ • X-MCP-Service-Token bypasses per-user rate limits │
│ • resources: config preview/status/snapshots, system │
│ • prompts: diagnose-traffic, security-review, ... │
└──────────────────────────┬───────────────────────────────────┘
│ ASGI in-process
▼
┌──────────────────────────────────────────────────────────────┐
│ api (FastAPI control plane) │
│ • all /api/v1/* endpoints become MCP tools │
└──────────────────────────────────────────────────────────────┘On startup, the backend auto-registers the mcp-server into the gateway (MCP_SELF_REGISTER=true), creating a corex-manager team, server, and skill with the operating guide. The gateway encrypts server auth secrets with MCP_SECRETS_KEY. Direct connections to mcp-server:8082 (bypassing the gateway) are gated by the optional COREX_MCP_TOKEN bearer token.
See MCP Gateway for connection snippets and the skill guide.
Data Persistence
DATA_DIR (default ./data, bind-mounted into containers)
│
├── haproxy/ ← shared haproxy-data volume
│ ├── haproxy.cfg generated HAProxy config
│ ├── coraza.cfg SPOE config
│ ├── coraza-spoa.yaml Coraza directives
│ ├── coraza-spoa.log WAF JSON event log (pruned)
│ ├── lists/ security list .lst files
│ ├── custom-rules/ remote WAF rule sets
│ ├── crs/ OWASP CRS snapshots
│ ├── api-armor/ schemas, keys, JWKS, profiles
│ ├── resp-transform/ response transform modules
│ ├── varnish/ VCL config
│ ├── vector/ vector.toml
│ └── mcp/ MCP config bundle + events log
│
├── certs/ ← TLS certificates + acme.sh state
├── postgres/ ← PostgreSQL data
├── valkey/ ← Valkey AOF/RDB (cache + token blocklist)
└── varnish/ ← Varnish storage fileThe haproxy-data volume is the shared substrate between the control plane and data plane — it’s how generated config reaches HAProxy without an HTTP call. PostgreSQL and Valkey persist their own data directories for state that must survive restarts.
Migrating from named volumes
If you have an older deployment using named Docker volumes instead of DATA_DIR bind mounts, see deploy.md in the corex_manager repo for the migration procedure. The DATA_DIR approach makes backups and inspection easier since all persistent data lives under one directory.
Next Steps
- Quick Start — Deploy with Docker Compose
- Docker Swarm — HA deployment on Swarm
- Kubernetes — Deploy via the Helm chart
- Security Rules — The coreX expression language
- WAF — Coraza SPOA + OWASP CRS
- MCP Gateway — Connect AI agents