The secure way
you query AI.

An open model port is a door with no lock.

KAILib can already point an app's model call at an Ollama anywhere — localhost, a LAN box, a machine on the web — by config alone. But Ollama's native API has no authentication and no TLS, and it exposes model management next to inference. Reaching a remote Ollama directly means plaintext prompts on the wire, an open door for anyone who finds the port, and no record that anything happened.

KQAI sits in front of [Ollama + model] and makes that hop secure, narrow, authenticated, encrypted, and auditable — without the calling app changing a single line.

Small enough to be safe. Three verbs, two transports.

Anything not on this list is not reachable — no model pull/delete, no config passthrough, no arbitrary Ollama API. That narrowness is the security property. The shapes mirror KAILib's own types, so the client maps 1:1.

One box. The model, boxed inside it.

The only port on the network is KQAI's TLS listener. Ollama is bound to loopback — nothing reaches the model except KQAI itself. And because the KAILib-side connector is just another LLMProvider, moving from a local Ollama to a fleet of KQAIs is zero app-code change.

From “hello” to a failover fleet.

Same KAILib surface throughout — client.ask(...) never changes. What changes is the box behind it.

Simple — a local sidecar

Stand a KQAI up in front of your own Ollama with one token. On the app side, point an AIBBProvider at it and call it like any model.

from kqai.config import KQAIConfig, Identity
from kqai.server import KQAIServer

cfg = KQAIConfig(
    tls_cert="cert.pem", tls_key="key.pem",   # TLS is not optional
    model="llama3.1",                          # the one model this box serves
    tokens={"workshop-secret": Identity("workshop-app", models=["llama3.1"])},
)
KQAIServer(cfg).serve()          # https://127.0.0.1:8443 — audited, on kai_lib

from kai_lib import LLMClient
from kai_lib.llm import AIBBProvider

client = LLMClient(AIBBProvider(
    "https://127.0.0.1:8443",
    token="workshop-secret",
    pin="sha256/…",          # pin the box's cert — no MITM
    model="llama3.1"))

print(client.ask("What's the torque spec for the caliper bolt?"))
# → "42 Nm" — over TLS, authenticated, and logged on the box

More complex — mTLS, quotas, and a failover fleet

Give each caller a client certificate and its own model allowlist + cap. Then compose several KQAIs behind one RouterProvider that prefers local → LAN → web and routes around a dead box on its circuit breaker.

cfg = KQAIConfig(
    tls_cert="box.pem", tls_key="box.key",
    model="llama3.1",
    ollama_host="http://127.0.0.1:11434",   # boxed — loopback only
    max_tokens=1024,                       # server-enforced ceiling
    tokens={
        "svc-dispatch": Identity("dispatch", models=["llama3.1"], max_tokens=512),
        "svc-report":   Identity("reporting", models=["llama3.1"]),
    },
)

from kai_lib import KAIApp, LLMClient
from kai_lib.llm import RouterProvider, AIBBProvider

def box(url):
    return AIBBProvider(url, client_cert="client.pem",
                        ca="ca.pem", pin="sha256/…")   # mutual TLS

router = RouterProvider([
    box("https://127.0.0.1:8443"),     # local sidecar
    box("https://ai-box.lan:8443"),    # LAN appliance
    box("https://infer.koperwas.com"),  # web endpoint
], strategy="prefer_local")          # local → LAN → web

app = KAIApp(llm=LLMClient(router))
print(app.llm.ask("hello"))
# pull the local box's plug → the LAN box answers. No app code changed.

Streaming — hands-free, token by token

The same secured channel, upgraded to a WebSocket for live tokens — the transport a voice/eyes-free client wants.

for tok in client.stream("Walk me through the failover procedure."):
    speak(tok)          # WSS /v1/stream — not cached, not retried mid-flight

What a locked door on the model makes possible.

Once the inference hop is authenticated, encrypted, and audited, whole classes of deployment that were “don't put that on the wire” become ordinary.