All projects
LLM InfrastructureActiveSince 2026
LLM Governance Engine
PII firewall and YAML policy engine that stops sensitive data before it reaches any LLM
About
After accidentally sending a real customer's credit card number to GPT-4o during a benchmark run, I built the LLM Governance Engine to make that class of incident impossible. It sits in front of every model call as a FastAPI dependency, scans prompts with Microsoft Presidio before they touch the wire, and either blocks (HTTP 403), warns, or fires a webhook — based on a YAML policy file you can hot-reload without a restart.
Key features
- Enforcement before the model call — not logging after the fact
- YAML policy DSL with hot-reload: no restarts to update rules
- Microsoft Presidio detects 50+ PII entity types (CREDIT_CARD, SSN, EMAIL, IBAN)
- PostgreSQL audit vault: every inference logged with PII flag, safety score, cost, latency
- CloudEvents webhooks to Slack, Teams, or PagerDuty on rule violations
- Multi-provider routing: OpenAI, Groq, Anthropic, Google Gemini, Ollama
Tech stack
FastAPIAngular 21PostgreSQLPresidioDockerSQLAlchemyFernet
I wrote a detailed post on why I built LLM Governance Engine and how it works under the hood.
Read the full story