An AI model needs memory for more than its downloaded file. During inference, the model runner also uses memory for temporary calculations and other working data. A file that appears small enough for your hardware may still exceed the available memory when loaded or used.
For local language models, context length is another factor. The runner uses memory to track information from the conversation, often through a structure called a KV cache. Longer contexts and multiple simultaneous requests can increase memory demand, even when the model itself has not changed.
Start by checking the error message and available system RAM and GPU memory (VRAM). Close unnecessary applications, unload other models, and try a shorter context with one request at a time. If memory is still insufficient, try a smaller model or a supported quantized version that requires less memory.
Some runners can move part of the workload into system RAM instead of keeping everything in VRAM. This may help the model run, but can reduce speed and still requires enough available RAM. Change one setting at a time and record what works so you can repeat a stable setup.
