Skip to content
On Prem AI Guide
Try Kortix
Four layers, four owners

On premise AI is four purchases. Kortix sells the open-source runtime.

The open-source AI Operating System

On premise AI keeps the model, the agent runtime and the record on machines your team operates, and a quote usually covers only the first. Kortix is the leading open-source alternative to Claude Cowork and ChatGPT Work.

Try KortixCompare the four layers

Self-host is free · One isolated sandbox per session · Roles, groups and an audit trail

Kortix desktop app, session view
Kortix desktop app, session viewkortix.com

Run it on Kortix Cloud, in your VPC, or on your own on-prem network. Self-host is free.

kortix self-host init --domain kortix.example.com --admin-email you@example.comkortix self-host startRun kortix self-host doctor and fix every failing check.Sessions run on Platinum sandboxes, not on the server.The instance must be reachable from the internet so those sandboxes can call back.
The layers

Four layers, one purchase decision each

Working note

Each layer has a product that fills it today, and a question only your team can answer.

  1. Inference

    The model server, and the GPUs under it

    Ollama runs open-weight models on a workstation and vLLM serves them at throughput on your own cards, while onprem.ai sells the same layer as a turnkey server.

    inference/gpu-plan.csv
    Model sizeWeights at FP16Cards to plan for
    8B16 GB1
    70B140 GB4

    Your call: which models, and who patches the server.

  2. Agent runtime

    Where the agent loop actually executes

    Dify and LangGraph give you the loop as something you build, ibl.ai hands you a sandboxed server per agent, and Kortix runs the open-source runtime each session lands on.

    agents/finance-close.md+3
    +permission:+  bash: ask+  ledger.write: ask

    Your call: where agent code runs, and on whose machine.

  3. Data and memory

    Where the files and the history sit

    The company memory and every file a session writes need a home on your side of the wall, in a repo or a bucket you already run.

    memory/close-2026-09.md
    WhatWho wrote itWhere it sits
    Close summaryagentyour repo
    Working filesagentyour bucket
    Transcriptsessionyour database

    Your call: what leaves with the transcript.

  4. Policy and audit

    Who can prove what an agent did

    Permissions and the record are the layer nobody quotes for and the first one an auditor asks about, so read what an audit needs before you sign.

    policy/agent-gates
    • shellask
    • ledger writeask
    • merge to maindeny
    • outbound httpblock

    Your call: who answers the auditor's question.

Side by side

The four layers, side by side

Kortix sells the runtime and the control layer. For a fully air-gapped build, onprem.ai and ibl.ai are the honest picks today.

KortixThe runtime
Inference
Not sold. Any model, your keys.
Agent runtime
One isolated sandbox per session.
Where state lives
In a git repo you own, on your server.
Policy and audit
Per-resource permissions for people and agents; roles, groups, audit trail.
Fully air-gapped
No. Images come from Docker Hub, sessions call back. No
Open source
Yes. Read it, fork it, audit it. Yes
onprem.ai
Inference
OpenAI-compatible APIs on Swiss-built servers, hardware or software.
Agent runtime
No agent runtime. Model serving and preconfigured apps.
Where state lives
On your servers; its page promises "no cloud dependencies".
Policy and audit
Controls "mapped to ISO 27001 and SOC 2", not certified.
Fully air-gapped
Yes, "even in air-gapped environments". Yes
Open source
No. No source published; nothing to read or fork. No
ibl.ai
Inference
Model-agnostic; local models if you want them.
Agent runtime
A sandboxed server per agent, inside your perimeter.
Where state lives
Behind your firewall, in code and data you own.
Policy and audit
"Every query is logged and auditable."
Fully air-gapped
Yes, on-premise or air-gapped, local models. Yes
Open source
No published source. You own what it builds, on its terms. No

What open source buys you

Four things your security reviewer can check without a call, and one line item that drops out of the budget.

  • Read it: The platform is source you can open and grep yourself.
  • Fork it: Change the runtime, keep the change in your own repo, and update on your own schedule.
  • Audit it: Your team reads the code that holds the data instead of a vendor questionnaire.
  • Self-host free: Put it on your own network and pay nothing for the software.

Kortix on GitHub

In plain terms

On premise AI, in plain terms

What does "on premise AI" mean?

AI served from hardware your own team runs, rather than from a vendor's endpoint. Where the compute stands is the term's whole scope; a real install also fixes where the runtime, the files and the audit record live. The full definition takes each layer in turn.

What does "on premise" mean?

Software installed on machines your team owns, in your building or a rack you rent, instead of on a vendor's cloud. The word covers the computer's address and says nothing about who wrote the code.

What is on prem vs SaaS?

On prem means the machines and their upkeep are yours; SaaS rents capacity from someone who owns it. The split repeats at each AI layer, so you can rent the tokens and keep the runtime and the log in your rack. Renting tokens or owning them is the cost side.

Can AI be on Prem?

Yes. The weights are downloadable, so Ollama or vLLM can serve a model from cards you own, and agent sessions can run on your network too. What limits the plan is GPU memory and someone to operate it, so start with server requirements.

Start with the layer Kortix runs

Self-host the runtime and the control layer, point it at the models you already serve, and keep the record on your side.

Try KortixSelf-hosting guide