As you read through the other pages in this area, you will come across a few technical terms. Here they are explained briefly, sorted by topic and with an everyday image, so that no word gets in the way.
This page explains the terms you encounter in the other articles of this area. Sorted by topic, not alphabetically, because related terms are more understandable together.
Five comparisons that carry the rest
Parameter
The figure 7B or 24B describes how many adjusting screws a model has. More adjusting screws mean more knowledge and finer answers, but also more space required. Comparable to the question of how many tools stand in a workshop. More tools mean more possibilities, but also a larger hall.
Quantisation
The model is stored at a coarser resolution, similar to a photo with lower colour depth. It looks almost the same but needs half the space. At usual settings the loss of quality is not noticeable for most tasks.
Context window
This much text the model can survey at once. Comparable to a desk: what does not fit on it cannot be taken into account in the answer. A model with a large context window reads a complete specification in one pass, a small one only individual chapters.
Document-based answer
Instead of letting the model answer from memory, you present it with the matching pages from your own archive. Like a clerk who looks in the file instead of guessing. That is the most effective lever against invented answers.
Determinism
A calculator always delivers the same result for the same input. A language model not necessarily, because many small computing steps run in a different order depending on load. This can be enforced technically, but it costs speed.
Models and basic terms
Language model (LLM)
A program that understands and generates texts because it has learned patterns from very large amounts of text. The basis of all applications described here.
Open weight
The finished trained file is freely downloadable, the training path is not disclosed. You can operate the model yourself, but not fully trace it.
Open source
Stricter definition, in which use and distribution are also unrestrictedly free. Many models advertised as open do not meet this.
Token
The smallest text unit a model processes. Roughly corresponds to a syllable. Context windows are stated in tokens.
Mixture of Experts (MoE)
The model activates only part of its building blocks per task. Saves computing time, but not memory. A common miscalculation in hardware planning.
Instruct model
Trained to follow instructions. First choice for summaries and evaluations of existing documents.
Reasoning model
Thinks in intermediate steps before answering. Good at calculating and weighing up, worse at pure summarising.
Multimodal
Processes text and image together. Necessary when fault images from quality assurance are to be evaluated.
Finetuning
Retraining with your own examples. Makes a model fit your technical language, but requires data and effort.
Operation and hardware
Inference
The ongoing operation, that is, the answering of requests. The cost block after acquisition.
VRAM
The working memory on the graphics card. Determines which model size loads at all.
Unified memory
With Apple, shared memory for processor and graphics unit. Allows large models on a single device.
Memory bandwidth
How fast data is read from memory. Determines the response speed more strongly than pure computing power.
Ollama
Simple program for downloading and running models. The usual entry point.
vLLM
Server software for high throughput and many simultaneous users. Necessary as soon as several departments access it.
GGUF
File format for downloaded models. Appears in the file name and shows the quantisation level.
Docker, container
Packaging that isolates an application together with its environment. Standard for productive operation.
WSL2
Linux environment within Windows. Recommended way if you operate productively under Windows.
Foundry Local
Microsoft's solution for local language models under Windows. Obvious if Windows Server is in use anyway.
Hugging Face
Central platform on which models are published. The source for original files.
Quality and reliability
Hallucination
The model invents plausible-sounding but false statements. The biggest quality risk in operation.
Grounding
The model may answer only from presented documents. The most effective countermeasure against hallucinations.
RAG
Procedure that searches for matching document passages and presents them to the model. Makes answers verifiable because the source reference comes with them.
Vector database
Store that finds documents by content similarity, not by keyword. Finds the right passage even with different wording.
Embedding
Conversion of text into numbers so that similarity becomes calculable. Technical basis of the document search.
Temperature
Control for the randomness of the answer. Low for facts, higher for draft texts.
Seed
Starting value for the randomness. Helps with repeatability, but does not guarantee it.
Benchmark
Standardised comparison test. Useful for pre-selection, does not replace a test with your own data.
Law and security
Apache 2.0
Permissive licence that requires only attribution and the disclosure of changes. The most uncomplicated case, even when passing on to customers.
Community licence
Manufacturer's own licence with additional conditions. Can be changed later, that is the actual risk.
GDPR
European data-protection law. Largely defused technically in local operation, because no processor arises.
EU AI Act
European AI law. Applies independently of the licence and requires different documentation depending on the risk class.
Risk class
Classification of an application under the EU AI Act into prohibited, high, limited or minimal. Determines the documentation effort.
NIS2
EU set of rules on cybersecurity, applicable law in Germany since December 2025. Affects, among others, mechanical engineering above a certain company size.
Prompt injection
Hidden instruction in a document that the model adopts. Relevant as soon as you automatically evaluate external documents.
Membership inference
Inference of whether a particular document is in the system. A confidentiality risk with sensitive holdings.
Zero trust
Principle of trusting no access without a check. Expressly recommended for language models by the BSI and the French ANSSI.
Least privilege
Everyone gets only the rights they need. Must be implemented in the data layer, not in the model.
Network segmentation
Separation of the network into sealed-off areas. Prevents the AI system from becoming a bridge between office and production.