redact — PII Redaction & Summarization Service

A self-contained service for extracting text from documents, detecting and redacting PII across multiple languages, summarizing the clean output, and exposing all capabilities to AI agents via MCP. All activity is traced to Azure Log Analytics Workspace.

Project structure

├── src/
│   ├── api.py                    # FastAPI app initialisation + router registration
│   ├── models.py                 # All Pydantic request/response DTOs
│   ├── constants.py              # Env var keys + hardcoded constants
│   ├── errors/
│   │   ├── exceptions.py         # Domain exceptions
│   │   └── exception_handlers.py # Global HTTP error handlers
│   ├── routers/
│   │   ├── health.py             # GET /health, /entities, /languages
│   │   ├── redaction.py          # POST /redact, /process
│   │   ├── summarization.py      # POST /summarize (quota enforced)
│   │   └── patterns.py           # GET|POST /patterns, DELETE /patterns/{id}, POST /patterns/test
│   ├── redaction/
│   │   ├── extractor.py          # File → plain text
│   │   ├── redactor.py           # Presidio PII detection + redaction
│   │   └── custom_patterns.py    # Per-business-unit pattern CRUD + persistence
│   ├── summarization/
│   │   └── summarizer.py         # Claude-powered summarization
│   └── util/
│       ├── auth.py               # API key dependency
│       ├── logging_config.py     # Structured logging + Azure Log Analytics handler
│       ├── rate_limit.py         # slowapi limiter + per-key identification
│       └── quota.py              # Daily LLM quota tracker (in-memory / Redis)
│
├── mcp_server/                   # MCP server — AI agent access layer
│   ├── app.py                    # Shared FastMCP instance
│   ├── server.py                 # ASGI app factory (SSE + auth middleware + /health)
│   ├── middleware.py             # MCP_API_KEY auth middleware
│   └── tools/
│       ├── redact.py             # redact_text, list_entities, list_languages
│       ├── summarize.py          # summarize_text
│       └── patterns.py           # list/add/delete/test custom patterns
│
├── tests/
│   ├── conftest.py
│   ├── unit/                     # Mocked, no external services needed
│   ├── integration/              # Full app via TestClient
│   └── utils/                    # Shared fixtures + mock data
│
├── data/
│   └── custom_patterns.json      # Persisted user-defined patterns (auto-created)
│
├── quota_config.json             # Per-team rate limits and LLM quotas
├── index.html                    # Single-page UI (no build step)
├── main.py                       # REST entry point shim → src.api:api
├── mcp_main.py                   # MCP SSE entry point → mcp_server.server:app (port 8001)
├── mcp_stdio.py                  # MCP stdio entry point → for Claude Desktop
├── Dockerfile                    # Single image, two startup commands
├── docker-compose.yml            # REST + MCP as separate scalable services
├── requirements.txt
└── .env.example

Quick start

1. Install dependencies

python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
python -m spacy download en_core_web_lg

2. Configure environment

cp .env.example .env
# Edit .env — set ANTHROPIC_API_KEY and optionally Azure + auth credentials

3. Run the REST API

uvicorn src.api:api --host 0.0.0.0 --port 8000 --reload

4. Run the MCP server (separate process)

uvicorn mcp_main:app --host 0.0.0.0 --port 8001 --reload

5. Open the UI

open index.html
# or: python -m http.server 3000 then visit http://localhost:3000

The UI expects the backend at http://localhost:8000.

6. Run both with Docker

docker compose up                        # start REST + MCP
docker compose up --scale mcp-server=3  # scale MCP independently

Architecture

Browser (index.html)                   AI Agents / Claude Desktop
        │                                        │
        │  REST (port 8000)                      │  MCP stdio / SSE (port 8001)
        ▼                                        ▼
FastAPI  src/api.py                    MCP Server  mcp_server/server.py
        │                                        │
        ├── src/routers/               ├── mcp_server/tools/redact.py
        ├── src/redaction/             ├── mcp_server/tools/summarize.py
        ├── src/summarization/         └── mcp_server/tools/patterns.py
        └── src/util/                            │
            ├── rate_limit.py                    │
            ├── quota.py                         │
            └── logging_config.py               │
                │                               │
                └──────── shared ───────────────┘
                       src/redaction/
                       src/summarization/
                       src/redaction/custom_patterns.py
                               │
                       Azure Log Analytics

Two transport layers, one business logic layer

Both MCP transports expose the same 9 tools backed by the same src/ services.

Request flow

Router / Tool → Service → (external call if needed)
    │                  └→ logger.info(..., extra={trace fields})
    │                          └→ AzureLogAnalyticsHandler → Log Analytics
    │
    ├── Rate limit check (slowapi — burst guard per API key)
    └── Quota check (quota.py — daily LLM calls per team)

Environment variables

Rate Limiting & Quotas

The service enforces two independent limits to prevent abuse and control LLM costs.

1. Burst rate limits (slowapi)

Applied per API key (falls back to IP when auth is disabled). Each key gets its own counter bucket — teams don't share limits.

When exceeded, the response is HTTP 429 with standard X-RateLimit-* headers.

2. Daily LLM quota (cost control)

Tracks Claude API calls per team per UTC day. Resets automatically at midnight — no cron job needed. Uses in-memory storage by default; set REDIS_URL for multi-pod deployments.

Quota response headers

Every /summarize response includes:

X-RateLimit-Limit: 10
X-RateLimit-Remaining: 7
X-Quota-Summarize-Limit: 50
X-Quota-Summarize-Used: 12
X-Quota-Summarize-Remaining: 38
X-Quota-Reset: 2026-04-06

Configuring per-team limits (quota_config.json)

Edit quota_config.json at the project root. Changes take effect on server restart.

{
  "default": {
    "requests_per_minute": 60,
    "process_per_hour": 30,
    "summarize_per_day": 50
  },
  "teams": {
    "their-api-key": {
      "name": "New Team",
      "requests_per_minute": 120,
      "process_per_hour": 60,
      "summarize_per_day": 200
    }
  }
}

Production — Redis backend

# .env
REDIS_URL=redis://your-redis-host:6379/0

Rate limit counters are then shared across all pods — no per-pod drift.

Testing rate limits

# Single call — inspect quota headers
curl -i -X POST http://localhost:8000/summarize \
  -H "Content-Type: application/json" \
  -d '{"redacted_text": "A person works at a company.", "length": "short"}' \
  | grep -E "X-Quota|X-RateLimit|HTTP"

# Python test — 6 calls against a quota of 3 (set summarize_per_day: 3 in quota_config.json)
python3 - << 'EOF'
import httpx, time
for i in range(6):
    r = httpx.post("http://localhost:8000/summarize",
        json={"redacted_text": "Test text.", "length": "short"}, timeout=15)
    used = r.headers.get("X-Quota-Summarize-Used", "?")
    rem  = r.headers.get("X-Quota-Summarize-Remaining", "?")
    lim  = r.headers.get("X-Quota-Summarize-Limit", "?")
    print(f"Call {i+1}: HTTP {r.status_code} | quota {used}/{lim} | remaining {rem}")
    time.sleep(0.2)
EOF

Expected output:

Call 1: HTTP 200 | quota 1/3 | remaining 2
Call 2: HTTP 200 | quota 2/3 | remaining 1
Call 3: HTTP 200 | quota 3/3 | remaining 0
Call 4: HTTP 429 | quota 3/3 | remaining 0   ← blocked
Call 5: HTTP 429 | quota 3/3 | remaining 0
Call 6: HTTP 429 | quota 3/3 | remaining 0

MCP Server

Overview

The MCP server runs as a separate process on port 8001 and exposes PII redaction capabilities to AI agents via the Model Context Protocol. It shares all business logic with the REST API — only the transport layer differs.

MCP tools

Authentication

# SSE transport (remote agents)
Authorization: Bearer <MCP_API_KEY>
# or
X-MCP-Key: <MCP_API_KEY>

The /health endpoint on port 8001 is always public (no auth required).

Claude Desktop setup

1. Open ~/Library/Application Support/Claude/claude_desktop_config.json
2. Add:

{
  "mcpServers": {
    "pii-redaction": {
      "command": "/path/to/project/.venv/bin/python3",
      "args": ["/path/to/project/mcp_stdio.py"]
    }
  }
}

1. Restart Claude Desktop — the 🔨 hammer icon will show all 9 tools under pii-redaction.

Remote agent setup (SSE)

{
  "mcpServers": {
    "pii-redaction": {
      "url": "http://mcp-server:8001/sse",
      "headers": {
        "Authorization": "Bearer your-mcp-secret-key"
      }
    }
  }
}

Example agent prompts (Claude Desktop)

Using pii-redaction, redact then summarize:
"John Smith (EMP-10029) submitted bid BID-2024-008821 from [email protected].
Contract CNT-2024-REF00812, SSN 123-45-6789."

Using pii-redaction, add a custom pattern for construction:
entity CONSTR_PROJECT_CODE, regex \bPROJ-[A-Z]{2}-\d{4}-\d{4}\b,
context "project code, proj id". Then redact: "Project PROJ-TX-2024-0091 assigned."

Custom PII Patterns

Per-business-unit regex patterns extend the built-in Presidio recognizers. Patterns persist to data/custom_patterns.json and activate on the next redact call.

Business units

Admin UI

Click Custom Patterns (gear icon) in the top-right header of the UI:

• Browse built-in patterns per business unit
• Add custom patterns with regex, context keywords, and confidence score
• Test regex against sample text before saving
• Delete user-defined patterns

API

# List all patterns for a unit
curl http://localhost:8000/patterns?unit=construction

# Add a custom pattern
curl -X POST http://localhost:8000/patterns \
  -H "Content-Type: application/json" \
  -d '{
    "entity_type": "CONSTR_PROJECT_CODE",
    "label": "Project Code",
    "unit": "construction",
    "patterns": ["\\bPROJ-[A-Z]{2}-\\d{4}-\\d{4}\\b"],
    "context": ["project code", "proj id"],
    "score": 0.85
  }'

# Test a regex
curl -X POST http://localhost:8000/patterns/test \
  -H "Content-Type: application/json" \
  -d '{"patterns": ["\\bPROJ-[A-Z]{2}-\\d{4}\\b"], "sample_text": "Project PROJ-TX-2024 approved"}'

# Delete a pattern
curl -X DELETE http://localhost:8000/patterns/{id}

Tracing & Logging

Every API action emits a structured trace record via Python's standard logging module.

Destinations

• Stdout — always active, human-readable. Captured by any container log driver.
• Azure Log Analytics Workspace — active when AZURE_LOG_WORKSPACE_ID and AZURE_LOG_WORKSPACE_KEY are set. Records land in PIIRedactionTrace_CL.

Trace fields

KQL queries

// All PII processing events in the last 24 hours
PIIRedactionTrace_CL
| where TimeGenerated > ago(24h)
| where event_type_s == "process"
| project TimeGenerated, client_ip_s, filename_s, detected_language_s, pii_count_d

// Top PII types detected this week
PIIRedactionTrace_CL
| where TimeGenerated > ago(7d)
| mv-expand entity_counts_s
| summarize total = sum(todouble(entity_counts_s)) by tostring(entity_counts_s)
| order by total desc

// Summarize quota usage per team key
PIIRedactionTrace_CL
| where TimeGenerated > ago(1d)
| where event_type_s == "summarize"
| summarize calls = count() by client_ip_s
| order by calls desc

API Reference

GET /health

{ "status": "ok" }

GET /entities

Returns all supported PII entity types including active custom patterns.

GET /languages

Returns supported languages and whether their spaCy NLP model is installed.

POST /process

Upload a file, extract text, auto-detect language, and redact PII.

Request — multipart/form-data

curl -X POST http://localhost:8000/process \
  -F "[email protected]" -F "language=auto"

Response

{
  "original_text": "My name is John Smith...",
  "redacted_text": "My name is <PERSON>...",
  "entities_found": [{"type": "PERSON", "start": 11, "end": 21, "score": 0.85, "original": "John Smith"}],
  "entity_counts": {"PERSON": 1},
  "detected_language": "en",
  "filename": "document.pdf",
  "file_type": "pdf",
  "session_id": "a1b2c3d4-..."
}

POST /redact

Redact PII from raw text.

curl -X POST http://localhost:8000/redact \
  -H "Content-Type: application/json" \
  -d '{"text": "Contact Sarah at [email protected]", "operator": "mask"}'

POST /summarize

Summarize PII-free text via Claude. Subject to hourly burst limit and daily LLM quota.

Returns HTTP 429 when daily quota is exceeded. Quota headers are included on every response.

GET /patterns

List custom PII patterns. Filter with ?unit=construction.

GET /patterns/units

List the four business units.

POST /patterns

Create a custom pattern.

DELETE /patterns/{id}

Delete a user-defined pattern. Returns 403 for built-in patterns.

POST /patterns/test

Test regex patterns against sample text without persisting.

Multi-language PII detection

Language is auto-detected via langdetect. The detected language drives spaCy NLP (names, locations, orgs). Country-specific IDs use regex and work regardless of which NLP model is installed.

python -m spacy download de_core_news_lg   # German
python -m spacy download fr_core_news_lg   # French
python -m spacy download pt_core_news_lg   # Portuguese

Scaling

Both services share the same Docker image. Scale them independently:

# Scale MCP server for more concurrent agent connections
docker compose up --scale mcp-server=3

# Scale REST API for higher HTTP throughput
docker compose up --scale rest-api=2

For Azure Container Apps:

az containerapp update --name mcp-server \
  --min-replicas 2 --max-replicas 20

Add REDIS_URL when running multiple REST API pods so rate limit counters are shared across replicas.

Development

# Run unit tests
pytest tests/unit/ -v

# Run integration tests
pytest tests/integration/ -v

# Run with coverage
pytest tests/unit/ --cov=src --cov-report=html

# Lint and format
ruff check --fix src tests mcp_server
black src tests mcp_server
isort src tests mcp_server

Interactive API docs

• Swagger UI: http://localhost:8000/docs
• ReDoc: http://localhost:8000/redoc
• MCP health: http://localhost:8001/health