FOUNDATION / 04
MODELS
Choose the right AI models for your hardware, goals, and creative workflows.
START WITH THE TASK
Choose the model based on what you want the system to do.
There is no single best local AI model. The right choice depends on the task, the quality and speed you need, and the memory available on your system. Start with the workload, then choose a model that fits your hardware.
MODEL CATEGORIES
Different models are built for different kinds of work.
A local AI studio will usually use several types of models rather than relying on one model for everything. Language models handle writing, reasoning, and tool use; image and video models generate visual content; multimodal models interpret multiple kinds of input; and specialized models support tasks such as speech, embeddings, upscaling, and segmentation.
01 / LANGUAGE MODELS
Writing, reasoning, coding, and tool use.
Large language models—or LLMs—power chat assistants, research workflows, code generation, document analysis, and MCP-connected agents. Model size strongly affects memory requirements, while quantization makes larger models practical on more modest hardware.
02 / IMAGE MODELS
Generate, edit, and refine visual content.
Image models can create new artwork, modify existing images, produce textures, explore concepts, and support design workflows. Their performance depends heavily on GPU speed and VRAM, especially when working at higher resolutions or using multiple conditioning tools.
03 / VIDEO MODELS
Create motion, animation, and generated footage.
Video models generate or transform sequences of images over time. They are among the most demanding local AI workloads, often requiring substantial VRAM, system memory, storage, and processing time to produce short clips at useful resolutions.
04 / MULTIMODAL MODELS
Understand text, images, audio, and other inputs together.
Multimodal models can analyze screenshots, describe images, interpret documents, answer questions about visual content, and connect different media within one workflow. They are especially useful for creative assistants, automation, and applications that need more than text alone.
05 / SPECIALIZED MODELS
Handle focused tasks within a larger workflow.
Specialized models support functions such as speech recognition, voice generation, embeddings, upscaling, background removal, depth estimation, segmentation, and pose detection. These smaller components often work alongside larger models to complete a production pipeline.
MODEL SIZE
Choose the smallest model that reliably handles the task.
Larger models can provide stronger reasoning, broader knowledge, and more capable output, but they also require more memory and run more slowly. A smaller, well-matched model often provides a better local experience than a larger model that barely fits into available VRAM or system memory.
PARAMETERS
Model size is commonly described by parameter count.
Labels such as 7B, 14B, 32B, and 70B indicate approximately how many parameters a language model contains. Parameter count helps estimate memory requirements, but it does not guarantee quality—a newer or better-trained smaller model may outperform an older, larger one.
QUANTIZATION
Reduce memory use while preserving useful model quality.
Quantization stores model weights at lower precision, allowing larger language models to run with less VRAM or system memory. Common formats such as 4-bit and 8-bit offer different tradeoffs between memory use, speed, and output quality. For most local systems, a good 4-bit quantization is a practical starting point.
CONTEXT WINDOW
More context allows the model to work with more information at once.
The context window determines how much conversation, documentation, code, or other input a language model can consider during a request. Longer context can be useful, but it also increases memory use and may reduce performance. Choose a practical context length for the task instead of automatically using the maximum.
MEMORY AND PERFORMANCE
A model must fit before it can run efficiently.
GPU VRAM is usually the fastest place to run a model, but system RAM can hold part or all of a language model when necessary. Offloading between GPU and system memory makes larger models possible, although performance drops as more work moves away from the GPU. Leave additional memory available for the operating system, applications, and model context.
8–12 GB VRAM
Start with efficient models and focused workloads.
This range works well for smaller quantized language models, many image-generation workflows, embeddings, transcription, and lightweight multimodal tools. Resolution, context length, batch size, and additional model components may need to be kept modest.
16–24 GB VRAM
Run stronger models and more complete creative workflows.
This range supports larger quantized language models, higher-resolution image generation, more demanding multimodal models, and workflows that combine several components. It offers a practical balance for a local AI workstation used for both creative production and general AI tasks.
32+ GB VRAM
Expand into large models and demanding production workloads.
Higher-memory GPUs can run larger language and multimodal models, more complex image pipelines, and some local video-generation workloads with fewer compromises. Additional VRAM also provides room for longer context, higher resolutions, larger batches, and multiple models loaded together.
TEST BEFORE YOU COMMIT
Evaluate models with your real workflow, not a leaderboard alone.
Benchmarks are useful for narrowing the options, but they cannot fully predict how a model will perform for your prompts, visual style, software, or production needs. Test a small group of candidates using the same inputs, settings, and hardware, then compare their quality, speed, memory use, and reliability.
TEST BEFORE YOU COMMIT
Evaluate models with your real workflow, not a leaderboard alone.
Benchmarks are useful for narrowing the options, but they cannot fully predict how a model will perform for your prompts, visual style, software, or production needs. Test a small group of candidates using the same inputs, settings, and hardware, then compare their quality, speed, memory use, and reliability.
QUALITY
Judge the output against the work you actually need.
For language models, compare accuracy, reasoning, instruction-following, writing style, and tool use. For visual models, evaluate composition, consistency, detail, prompt adherence, editability, and how well the output fits your intended production pipeline.
SPEED
Measure whether the model is responsive enough for the workflow.
A slower model may be acceptable for overnight rendering or occasional analysis but frustrating for interactive writing, coding, or creative iteration. Record generation time, token speed, loading time, and how performance changes when other studio applications are running.
COMPATIBILITY
Confirm that the model works with your chosen software.
Check that the model format, quantization, architecture, and required extensions are supported by your local runtime or creative application. A highly rated model provides little value if it cannot load reliably, use the required tools, or connect to the rest of your studio.
LICENSE AND USE
Verify what the model allows before using it in production.
Review the model license, usage restrictions, attribution requirements, and commercial-use terms before building it into a workflow or product. Also check the licenses of fine-tunes, adapters, and supporting assets, since they may differ from the original base model.
FIND AND ORGANIZE MODELS
Use trusted sources and keep every download identifiable.
Download models from established repositories, official developer pages, or the library built into your chosen runtime. Record the model name, version, format, quantization, source, and license so you can reproduce the setup later and avoid confusing similar files.
TRUSTED SOURCES
Start with the model creator or a well-established repository.
Hugging Face is the primary repository for many open models, while tools such as Ollama provide curated libraries in ready-to-run formats. When possible, follow links from the original model developer and verify the publisher, documentation, file history, and community activity before downloading.
MODEL FORMATS
Match the file format to the software that will run it.
Language-model tools may use formats such as GGUF, Safetensors, or runtime-specific packages. Image and video systems commonly use Safetensors along with separate encoders, VAEs, adapters, or control models. Check the application’s documentation before downloading large files.
STORAGE
Organize models so they can be updated, moved, and reused.
Keep models in clearly named folders organized by type, developer, and version. Separate original downloads from custom fine-tunes and workflow-specific files, and avoid keeping unnecessary duplicates. Large model libraries can consume storage quickly, so track what is actively used.
MODEL SELECTION WORKFLOW
Move from the task to a tested local model.
Define the task and the required output.
Identify the software and model formats that support it.
Set a memory limit based on available VRAM and system RAM.
Select a small group of appropriately sized candidates.
Test them with the same inputs and settings.
Compare quality, speed, memory use, reliability, and licensing.
Keep the best-performing model and document the configuration.
BUILD A STARTER LIBRARY
Begin with a few dependable models instead of collecting everything.
Start with one general-purpose language model, one coding-capable model if needed, one image-generation system, and only the specialized models required by your immediate workflow. Learn their strengths and limitations before adding alternatives. A small, documented library is easier to maintain than a large collection of untested downloads.
NEXT STEP