Questions & answersCan you run ChatGPT locally in a company?
The question comes up in almost every first call. The honest answer has two parts: no, not ChatGPT. Yes, what you want from it.
Not ChatGPT itself, which is a cloud service by OpenAI and cannot be installed in your own data centre. What can run locally are open language models with an interface that works like ChatGPT. For summarising, writing, translating and questions about your own documents they are sufficient in daily use, and no request leaves the company.
The term behind it is explained in the primer What is a local LLM? This article is about the practical side: what works, what does not, what it takes.
When does it make sense?
When employees work with texts, documents and cases that must not go to a foreign cloud. Contracts, HR data, engineering documents, customer correspondence. As soon as the data protection officer has prohibited or restricted the use of ChatGPT, a local model is the way to offer AI anyway. It also makes sense when many users work regularly and the running costs of user licences and tokens should become predictable.
When does it not make sense?
If only a few people occasionally edit non-critical texts, a cloud account with a processing agreement is faster and cheaper. A local model also does not make sense if the application needs the very strongest models for complex reasoning and the data is non-critical. And anyone without an IT team that can look after a server or device should start with a European cloud.
Prerequisites
- Hardware with enough GPU memory: a compact device with an AI accelerator for a team, a server with a GPU for many users.
- A data classification that defines which documents may enter the system.
- An IT team that can look after a server or virtual machine. A service provider does the setup, operations stay in-house.
- A permissions concept: who may see which knowledge base, who may switch models.
Options
With Open WebUI and Ollama, employees get an interface that feels like ChatGPT: chat with history, several models to choose from, login via the company directory. On top of that comes what ChatGPT does not offer without additional services: questions about your own documents with source references, fixed roles and permissions, and a connection to workflows so the AI can also work in the background.
Benefits
- No request and no document leaves the company.
- No usage fees: one-off hardware, then unlimited use.
- Free choice of model, including European models, and the option to switch.
- Shadow AI with private accounts is replaced by an approved offering.
Limits and risks
Open models of medium size are weaker than the largest cloud models for complex, multi-step tasks. Response speed depends on the hardware. And local does not automatically mean secure: without a permissions concept and purpose limitation, an internal system is as vulnerable as any other. Model updates have to be planned deliberately, they do not arrive by themselves.
Example
A supplier with about 120 employees had banned the use of ChatGPT because quotations and drawings showed up in requests. Usage continued anyway, privately and invisibly. An internal AI portal with a local model, login via the directory and a knowledge base for work instructions solved the problem: employees got a tool they were allowed to use, and IT could see what happened with it. The pilot ran for four weeks on an appliance, then the system moved to an in-house server.
Frequently asked
Is a local ChatGPT alternative as good as the original?
For most everyday business tasks, yes: summarising, structuring, translating, classifying and questions about your own documents. For very long, multi-step reasoning tasks the largest cloud models are ahead. For confidential data the difference does not matter because the cloud is not an option there anyway.
Which software do I need for ChatGPT locally?
A model server such as Ollama that loads open language models, and an interface such as Open WebUI that provides chat, history, user management and document questions. Both are open source and run on Linux, macOS or Windows.
How many users can a local installation serve?
A compact device with an AI accelerator serves one team. For dozens of concurrent users you need a server with one or more GPUs. The limit is GPU memory and the number of simultaneous requests.
What about data protection and GDPR?
Local operation avoids transfers to third parties and with them the hardest GDPR questions. The system becomes compliant through a permissions concept, purpose limitation and documentation. That is part of a proper rollout.
Can I test this beforehand?
Yes. A pre-configured appliance arrives, is connected to the network and runs the same day. After four weeks with real documents you know whether local AI holds up for your cases.
Conclusion
ChatGPT stays in the cloud. The function companies want can be replicated locally, with open models and open software. Whether that is the right path is decided not by the technology but by data class and number of users. A four-week test with real documents answers the question better than any presentation.
Related questions: Open WebUI or ChatGPT: what fits a company?, What does a local enterprise AI cost?, What is a local LLM? And why is it relevant for organisations?
Local AI for business
Language models on your own hardware, GDPR-compliant and without vendor lock-in. Options, comparison with the cloud and the path to a pilot.
To the solution: local AI