Questions & answersWhich open-source AI is suitable for companies?
Not one model, but a stack. And a rule by which to choose the components.
For companies, a stack of four open components is suitable: Ollama as model server, Open WebUI as interface, ChromaDB as knowledge base and n8n for automation. As language models, open models from Mistral, Meta, Qwen and others come into question, selected by licence, size, language and context window. Which model concretely fits is decided by the use case, and that can be checked in a pilot within a few weeks.
When does open-source AI make sense?
When data should stay in-house, because open software can be run locally. When licence costs for many users should be avoided. When model choice should stay free, including European models. And when independence from a provider is a goal, because open components can be swapped.
When does it not make sense?
When nobody can or wants to operate the software. Open source is free in licence, but not in operations. If an application absolutely needs the very strongest models and the data is non-critical, a cloud model is the shorter route. And anyone who only wants to try things out should not start by building a stack, but with a ready-made appliance.
Prerequisites
- Hardware with enough GPU memory for the chosen model size.
- An IT team that looks after servers or virtual machines and applies updates.
- A check of model licences for commercial use.
- A data classification that justifies the operating path.
Options: the stack
- Ollama. The model server. Loads open language models, runs them on the hardware and provides an API. Models can be swapped at any time.
- Open WebUI. The interface for employees: chat, model selection, knowledge bases, roles, login via the directory.
- ChromaDB. The vector database for company knowledge. Holds document sections so that matching passages for a question can be found.
- n8n. Workflow automation. Connects the AI with email, ERP, CRM and file storage, for automations and agents.
- Open language models. From Mistral, Meta, Qwen, Google and others, in various sizes and quantisations. Selected by licence, size, language, context window and task.
Benefits
- No licence costs for the core components.
- Can run locally, without data leakage.
- Every component replaceable, no lock-in.
- Large communities, fast development, plenty of documentation.
Limits and risks
The projects develop quickly, updates come often and occasionally break something. Model selection is a moving target: what is the best model of a size class today is no longer in six months. And open source shifts responsibility: what a provider does for you in the cloud, you or your service provider do yourselves.
Example
A trading company with 200 employees wanted AI for everyone, without data leakage and without a licence per user. The stack of Ollama, Open WebUI and ChromaDB ran on a server with one GPU, connected to the directory. The model was chosen after a test with tasks from sales and procurement, a medium-sized European model was enough. After half a year it was swapped for a newer generation without touching the interface or the knowledge bases.
Frequently asked
May open language models be used commercially?
Most yes, but the licence has to be checked per model. Some allow free use, others have conditions above a certain company size. That is part of the model selection.
Are open-source models as good as ChatGPT or Claude?
In everyday business use for summarising, structuring, translating and document questions, on par. For complex reasoning the largest cloud models are ahead. The gap has narrowed in recent years.
Which model should I choose?
That depends on task, language, data class and hardware and changes with every model generation. That is why I do not recommend a model on a website but after a test with your tasks. The selection criteria are stable, the names are not.
Is open source secure?
Open code can be reviewed, and the large projects are watched closely. But a system becomes secure through operations: updates, access protection, permissions. That applies to open and closed software alike.
Is there support for open-source AI?
For Open WebUI and n8n there are commercial offerings from the projects. For the whole stack, service providers like me handle setup, training and support. Operations stay with your IT.
Conclusion
Open-source AI for companies is a stack of model server, interface, knowledge base and automation, with an open language model chosen by criteria rather than by name. Anyone who can operate the software gets data control, predictable costs and independence. Anyone who only wants to test starts with an appliance on which the same stack is already running.
Related questions: Mistral, Microsoft and the question of who owns your data, Which hardware does a local LLM need?, Open WebUI or ChatGPT: what fits a company?
Local AI for business
Language models on your own hardware, GDPR-compliant and without vendor lock-in. Options, comparison with the cloud and the path to a pilot.
To the solution: local AI