One secure gateway to all your AI models, local and cloud. The gateway automatically detects sensitive data, masks it, blocks prompt injections, routes every request to the right model and keeps an audit log of every request. Your teams simply use AI. Your data stays inside the walls.
The pipeline
Every request passes through five automatic checks before it reaches a model. This is no brochure flowchart: it is the exact order in which the code runs them.
Emails, phone numbers, card numbers, passwords, API keys, public IP addresses, detected with regex patterns and semantic keywords. A hit? The request is forced to a local model, whatever you asked for.
"Ignore all previous instructions" and its friends are detected in user messages. Such an attack is never forwarded: the request is handled locally and safely at once.
Every request lands on the right model: explicit tags, configurable regex routes and, if nothing matches, semantic similarity via embeddings. Your rules, live-adjustable, without a restart.
Sensitive data is masked in the prompt itself before sending: [EMAIL], [PHONE], [CRED]. Internal IP addresses stay readable, so technical questions remain legible.
Every request gets a unique ID (or inherits your X-Request-ID). Routed model, classification, latency, token usage and masked text: fully traceable, append-only.
Defense in depth, built in: suppose a route points at a cloud model after all, and the request does contain sensitive data: the request is then forced back to a local model at the very last moment. Sensitive content simply never reaches the cloud. Period.
Features
One endpoint to all your models, with a security layer on by default, not as an option.
Your teams, scripts and agents talk to one OpenAI-compatible address. Behind that single door sit your local models (Ollama) and, if you allow it, carefully chosen cloud models. Switching models? One line in the config; no client notices a thing.
Every request automatically goes to the best model: coding questions to your coding model, writing to your writing model, sensitive content always local.
Credit cards, emails, phones, keys, passwords, public IPs: all detected and masked, in the prompt and in the logs alike.
Jailbreak attempts and "ignore all instructions" tricks are detected and kept out of the cloud, automatically, without you lifting a finger.
A model degrades or drops? The gateway automatically switches to your fallback chain, with retries and cooldowns. Your app notices nothing.
Append-only log of every call: masked input & output, routed model, why that route, latency and token usage, all correlatable via request ID.
A detection engine that watches LLM and auth logs live. Define your own use cases with a simple match DSL; alerts go to the UI and to your webhook.
Zero-dependency authentication, built the way it should be: scrypt hashing, HMAC sessions, optional TOTP 2FA, service keys for agents, and lockout with exponential backoff.
Live monitor, playground to test routes before you roll them out, usage dashboard, rules editor with classification test: no external service, just runs at your place.
Under the hood
Every event is written append-only, readable with any tool you already have. This is a real log line, as it comes in:
Note: the client had asked for the fast cloud model. The gateway detected the sensitive data, masked it in the log, and routed the request to the local private model instead. The client noticed nothing, except that it just worked.
Specifications
| API protocol | OpenAI-compatible (chat-completions, streaming) |
| One endpoint | Your entire model fleet behind one URL |
| Authentication | Master key + service keys per agent/app/script |
| Request-ID tracing | Inherit X-Request-ID or automatic |
| Backends | Ollama (local) + optionally permitted cloud models |
| Routing layers | PII → tag → regex rules → semantic (embeddings) → default |
| Semantic router | Local embeddings, threshold & examples in the UI |
| Failover | Per-model fallback chains, retries, cooldown |
| PII | Regex + keywords + phone + public IPs; masking in prompts and logs |
| Injection | Detected jailbreak patterns → forced local |
| Defense in depth | Sensitive + cloud route → last resort: local model |
| Auth | Scrypt + HMAC sessions + optional TOTP + lockout backoff |
| Dashboard | Rules editor, live monitor, playground, usage, SIEM |
| Live reload | Firewall rules & SIEM use cases without restart |
| Log format | Append-only JSONL, with request-ID correlation |
| Webhooks | SIEM alerts to your own endpoint |
| Installation | Docker Compose, entirely on your hardware |
| Core components | LiteLLM proxy with routing hook + zero-dependency Node UI |
| Dependencies (UI) | None. No npm packages, no CDNs |
| Cache | Disk cache + semantic route cache |
| On-site installation | We set it up, anywhere in Belgium |
| Training | Your team learns to manage the gateway itself |
| Support | Optional management plan, or simply standalone |
| Ownership | The code runs at your place. Full stop. |
In the live demo we run your real test questions through the gateway, so you can see with your own eyes how PII is detected, masked and kept local. No obligation.
Which models do you use today, and where does privacy hit a wall?
Your routing rules, your models, your users, in a concrete proposal.
On your hardware, on your premises. Tested with your real use cases.
You learn to manage the gateway yourself; we stay a call away.