LLM keys & providers

For org admins.

Hekkos runs every LLM call — the PR doc-impact classifier, the doc/section writer, and RAG (retrieval-augmented generation) chat answers — through a provider. This guide covers the three ways that provider is chosen, how to bring your own key, and the optional answer-quality cascade.

The three modes

Mode	Who uses it	Where the key lives	Billed to
Pooled (managed)	SaaS default — no setup	Hekkos’s shared provider keys	Your Hekkos plan (token budget)
Bring your own key (BYOK)	SaaS, opt-in per org	Encrypted in Hekkos, set in the dashboard	Your provider account
Self-hosted / deployment	`DEPLOY_MODE=self-hosted`	Your deployment’s environment	You (you run the infra)

You don’t have to bring a key — pooled is the default and needs nothing. BYOK is for orgs that want to run on their own provider account (and their own rate limits and billing).

Bring your own key (SaaS)

BYOK is off until the deployment operator turns it on with the BYOK_MODE env var (single for one org-wide key, per_role to also expose per-role model slots; the default off hides the LLM-key UI entirely and runs every org on the pooled default). The rest of this section assumes it’s enabled.

An admin sets this on Settings → LLM Configuration:

Provider — Groq, Anthropic, or OpenAI (hosted providers only; see Security for why).
Model — chosen from that provider’s supported models.
API key — pasted once. It’s encrypted with AES-256-GCM before storage and is never returned to the browser again (you’ll see a “Set” badge).
Test key — runs a tiny live probe against the provider and shows a green “validated” or a red reason (e.g. “key rejected by provider”), so you know the key works before you rely on it.
Save.

You can also set this up during onboarding: when the deployment offers BYOK, the setup wizard’s Preferences step shows an AI models row (marked optional) whose Customize expander takes the same provider, model, and API key — or skip it there and add a key later in Settings.

Once saved, your key powers every LLM call for your org on the model you selected. If your key is ever rejected by the provider (revoked, out of quota), Hekkos automatically falls back to the pooled key for that call, so a bad key degrades gracefully rather than breaking chat — you’ll see the last-checked status turn red on the settings page.

One model, or one per role. In BYOK_MODE=single a BYOK org runs all of its LLM roles (classifier, writer, RAG) on the single model you pick. In BYOK_MODE=per_role the settings page also exposes per-role slots — classifier, doc-writing, and chat — so you can, for example, run the cheap/high-volume classifier on a smaller model and reserve a stronger one for chat. A slot you leave unset falls back to your org-wide key, then to the pooled default. See local models.

Removing a key

Clear the key (leave the field blank and turn off BYOK, or use the remove action) — the org reverts to the pooled default on the next call.

Answer-quality cascade (optional)

RAG answers are usually fine on a cheap model, but hard questions benefit from a stronger one. The cascade lets you run answers on a cheap primary model and escalate only the weak ones to a premium model — so you pay premium prices only when they’re needed.

On Settings → LLM Configuration, enable “Escalate weak answers to a premium model” and pick the premium model (same provider as your primary). Then:

Hekkos answers on your primary model first.
If that answer looks weak — empty, a refusal (“I don’t know…”), or grounded in poorly-matching context — it re-runs the question once on your premium model, on your own key.
If the escalation fails for any reason, you keep the primary answer. It never fails a query, and a question is billed once.

This is off by default. The “weak answer” thresholds are conservative defaults that a deployment operator can tune.

Self-hosted / deployment keys

In DEPLOY_MODE=self-hosted, keys are not set per-org in the dashboard — they come from your deployment’s environment, because you run the whole stack. The settings page shows the LLM fields read-only with the source marked “Deployment,” so members know where the config lives.

The env vars themselves are operator material: see the configuration reference for every variable, and local models for running roles on a bundled Ollama.

Security

Encryption. A BYOK key is stored only as an AES-256-GCM ciphertext (internal/crypto, keyed from MASTER_KEY), decrypted in memory at call time, and never returned by any API, logged, or placed in an error message.
Hosted providers only (SaaS BYOK). The provider list is a fixed allowlist (Groq / Anthropic / OpenAI) with known endpoints — you can’t point a SaaS BYOK key at an arbitrary URL, which would be an SSRF vector. Local Ollama URLs are a self-hosted-only concern, where you own the network.
Per-org isolation. Every key read is scoped to a single org; one org’s key is never visible to, or usable by, another.

Local models — running the classifier/other roles on a bundled Ollama, in either deploy mode.
Plans & usage — token budgets and quotas (a BYOK org still meters usage for visibility; the spend is on your provider account).