Local AI on PC: what it is, what it's for, and how to launch it

What is local AI — in plain English

Local AI is artificial intelligence that runs on your computer. You install a program, download the required model, and communicate with it in a chat window. Your PC performs the computations for the answers.

A standard cloud chat works differently: you send a message via the internet, a company's server processes it, and an answer is returned to you. For local execution, your processor or graphics card takes over this work. After installation and downloading the model, the basic chat can even work without the internet. [1]

Imagine a music player: you can listen to music online, or you can download it to your computer. AI works on a similar principle, except instead of a song, you download a model — a set of files that allow the program to understand the request and generate a response. There is no need to train it from scratch.

What it's for: examples from everyday work

A local model is useful when you frequently work with text, files, or code and want to have your own assistant at hand. Here is where you can start.

The table can be scrolled sideways →

For example, you can give the model your own work notes and get a clear plan. Or ask it to explain an unfamiliar script before you run it. If the program supports document processing, you can add the file directly to the chat. In LM Studio, such processing of uploaded documents can be done locally. [1]

Important boundary: a downloaded model won't find the latest news on its own. For current prices, events, and finding sources, you need internet access or materials provided by you. And the model's answers may still contain errors.

Local AI or cloud: what is more convenient for you

The table can be scrolled sideways →

The local option is worth trying, if you need to work without a network, want to keep documents on your PC, or regularly perform the same simple tasks. It is also interesting for custom programs, experiments, and choosing the assistant's behavior.

Cloud chat is often more convenient, if you need a powerful model without setup, and your computer is weak. You can combine both approaches: process daily notes locally, and send complex requests to a more powerful cloud model.

For private work, also check the program's extensions: web search, voice service, or connected tools may access the internet. The mere fact of installing a program on a PC does not mean all its functions are local.

First run without code: LM Studio

LM Studio is a program where you can find a model, download it, and start a chat. This is enough for a first introduction. [2]

1. Download LM Studio from the official website. Choose the version for Windows, Linux, or your Mac and install it.
2. Open model search. In the documentation, it is called Discover. Find Qwen3.5-4B. If you only have 8 GB of RAM, start with a smaller one Qwen3.5-2B.
3. Choose a file to download. If you see several options, look for a compatible GGUF version marked with approximately Q4 to start. This is a compressed model that requires less memory. Do not download all options at once.
4. Wait for the download to finish and open Chat. Select a model from the list and click Load into memory — Load, if your version shows such a button.
5. Write a simple request in Ukrainian. For example: "Here is my letter. Shorten it by half, keep all dates and amounts, and make the tone friendlier."

The first run may take more time: the program reads large files from the disk. Subsequent responses will come from the loaded model. If it doesn't fit or works very slowly, choose a smaller size. [4]

In newer interfaces, you might see the name Bionic and cloud model suggestions. For the described route, choose specifically the model whose files are downloaded to your computer.

Is your computer enough

To start, you only need to know two numbers: how much RAM your PC has and how much dedicated VRAM your graphics card has. In Windows, you can find this in Task Manager → Performance.

• RAM is your computer's random access memory. It is used by Windows, open programs, and the model.
• VRAM — the dedicated video card memory. When the model fits in it, the card can perform the main work faster.
• SSD is storage for files. It is needed to load the model, but a large SSD does not replace RAM or VRAM.

If Windows shows additional "shared" memory for the graphics card, this is part of the regular RAM. It doesn't make an 8 GB card the same as a card with 24 GB of dedicated memory. For laptops, also check the exact configuration: mobile and desktop cards with similar names can differ.

The table can be scrolled sideways →

For LM Studio on Windows x64, you need a CPU with AVX2 support; 16 GB RAM or more is recommended. On a very old PC, check the software compatibility first. [3]

NVIDIA and Radeon examples to find your level

RTX 3060 comes with 8 or 12 GB of memory. RTX 4060 Ti has 8 or 16 GB. Radeon RX 7600 XT has 16 GB, RX 7900 XTX has 24 GB; the desktop RTX 4090 also has 24 GB, and the RTX 5090 has 32 GB. The professional RTX PRO 6000 Blackwell Workstation Edition has 96 GB. These are examples of capacity, not a speed ranking or a list of must-buys. [40]–[45]

Advice before spending: first run a small model on the PC you already have. Then it will become clear what exactly is lacking: speed, memory, or response quality. More memory allows you to run a larger model, but it doesn't fix its errors by itself.

Which model to choose: simpler and more powerful

Imagine a model as a specific assistant that you run in a program. You can download several and compare them on a single task. There is no need to install everything.

A number with a letter B usually means billions of parameters. For your first choice, a simple rule is enough: smaller models are easier to run, while larger ones require more memory and can perform complex tasks better. But a larger model won't always be better for your specific work.

The table can be scrolled sideways →

If you don't want to spend a long time choosing: start with Qwen3.5-4B. If it works comfortably but the answers are too simple, try 9B. On a powerful PC, compare Qwen3.8-27B with Gemma 4 26B or 31B. Check Ukrainian language support, accuracy, and usefulness with your own examples — there is no universal winner for all tasks.

What Q4 means during download

One model can be offered in several compressed versions. This is called quantization. Q4 usually takes up less memory, Q8 takes more but preserves the precision of the original weights better. For a start, Q4_K_M is often chosen if that version is available and supported by the program. GGUF is simply the name of the file format. [4]

Look at the size of the specific download. An 8 GB file does not guarantee that the model will need exactly 8 GB of memory during operation: additional space is needed for the conversation itself and computations.

NVIDIA and AMD Radeon: how to run without unnecessary complications

If you have NVIDIA

Update the driver from the official NVIDIA website, install the current software for models, and start with a small model. Ready-to-use tools like LM Studio or Ollama usually don't require you to manually install the entire CUDA developer toolkit.

The program must detect the card and use it for computations. If the card is supported but the model doesn't fit, some of the work may go to the CPU — in that case, the response often slows down. During the test, close games, renders, and other heavy programs. If you are on a laptop, plug in the power. [19]

If you have a Radeon

Local AI also works on Radeon. AMD documentation provides a ready-made path via LM Studio. However, support depends on the specific card, operating system, and software version. [20]

You might encounter names like ROCm, HIP, and Vulkan. These are ways the program uses the graphics card. For your first run, you don't need to study them as separate disciplines: check the instructions for your card and select the supported option in the program. For Ollama on Windows and Linux, there is a path via Vulkan; ROCm support lists for these systems differ. [19] [21]

The order is simple: find your exact card → check compatibility → update drivers and software → download a small model → see if the GPU is being used. For AMD, do not blindly copy instructions written for CUDA/NVIDIA.

If you don't have a dedicated graphics card

Try a small model on the CPU. Integrated graphics can also help if the program supports them. This is a normal way to understand if you even need local AI at all before spending money on hardware.

On Mac with Apple Silicon, programs can use Metal or MLX. There, memory is shared between the system and graphics, so you also need to leave some headroom. An 'AI PC' label or a high TOPS rating on a new laptop doesn't guarantee running any model: compatibility with the specific program is required. [3] [22]

What's free, what costs, and is there a limit

A basic local chat can be run without a subscription. Download a free program and an accessible model — and you're working on your own PC. You don't have to pay a cloud service for every response.

There are still costs: electricity, disk space, and the hardware itself. If you already have a computer, a separate purchase is often not needed for the first test. If you connect a paid voice, search, or cloud model, those services are paid for separately.

The table can be scrolled sideways →

"Unlimited" here means the absence of a cloud service daily quota for local requests. But there are limits to your PC: memory, speed, heating, and the amount of text the model can process simultaneously. A paid subscription to cloud features does not add video memory to your computer.

And what does "uncensored" mean?

Some authors of modified models write 'uncensored' or 'no refusals.' This usually means the model is less likely to refuse to answer certain topics. This is not a guarantee of smarter answers, a complete absence of refusals, or better Ukrainian. For practical use, it's more useful to check accuracy than to choose based solely on that name.

For commercial use, check the license of the specific model and program. The listed official Qwen3.5, Qwen3.8-27B, Gemma 4, and gpt-oss are noted as Apache 2.0; for modified versions and other families, the terms may differ. [6] [9] [10] [12]

Choose where to start on your PC

Select the specifications below. The selection will provide initial advice; check the exact compatibility of your card and software in their respective instructions. It does not scan your computer.

Can I create images, voice, and video locally?

Yes, but different tasks require different models. A text model can write a script, but that doesn't mean it can draw a frame or voice a line.

• Images and image editing. A common option is ComfyUI with a ready-to-use workflow. For example, there is the more compact FLUX.2 klein 4B and the newer local workflow FLUX 3 Image Klein. You need to download all files specified in the workflow and check the memory requirements. [33]–[36]
• Speech-to-text. Buzz provides a convenient interface, while whisper.cpp offers more technical control. For Ukrainian, choose a multilingual model rather than an English-only one. [37] [38]
• Text-to-speech. There are local voices like Piper, including Ukrainian. It's worth listening to the accent and sound quality with your own text. [39]
• Video. You can run compatible video models via ready-made workflows, but requirements depend on duration, resolution, and the model. A computer that is good for text might not necessarily generate video quickly.

ComfyUI has both local models and nodes that connect to paid services. If you need to work offline, choose a workflow with local files. Generator licenses also vary: FLUX.2 klein 4B and FLUX 3 Image Klein have different terms.

For a voice character, there is a separate guide with character, memory, and a Ukrainian voice. For creative work — a guide to creating an AI cartoon or film.

For pros: how to connect a local model to your workflow

If a simple chat is already working, you can move further: connect the model to a code editor, your own application, or document processing. Below is the technical part. These settings are not mandatory for casual conversation.

Ollama: running via commands

Ollama is convenient when you want to manage models from the terminal and connect other programs to them. Install it from the official website, open your terminal or PowerShell and run the example below. On the first run, the model will need to be downloaded from the internet. [23]

ollama --version
ollama run qwen3.5:4b

# For a weak PC: ollama run qwen3.5:2b # Next step: ollama run qwen3.5:9b # For a station with sufficient memory: ollama run qwen3.8:27b ollama run gemma4:31b ollama run gpt-oss:20b

# A separate terminal window while the model is running: ollama ps # List of already loaded packages: ollama list # Unload from memory: ollama stop qwen3.5:4b # Only if you want to delete the package from the disk, remove the #: # ollama rm qwen3.5:4b

What is a local API

An API is a way for one program to communicate with another. For example, your code editor sends a request to a local AI and displays the response within itself. The model can remain on your PC during this process.

The OpenAI API-compatible address for Ollama is — http://127.0.0.1:11434/v1. The term 'OpenAI-compatible' refers to the communication format between programs, not a requirement to send requests to OpenAI. In LM Studio, the server is turned on via Developer → Start server; get the address from its settings. [26] [27]

import json import urllib.request payload = { "model": "qwen3.5:4b", "messages": [{ "role": "user", "content": "Explain the difference between RAM and SSD in Ukrainian in three sentences." }], "stream": False, "options": {"num_ctx": 4096, "num_predict": 256} } request = urllib.request.Request( "http://127.0.0.1:11434/api/chat", data=json.dumps(payload).encode("utf-8"), headers={"Content-Type": "application/json"}, method="POST" ) with urllib.request.urlopen(request, timeout=180) as response: result = json.load(response) print(result["message"]["content"])

Initially, keep the server accessible only on your computer. If other people or devices will be connecting, configure access separately. LM Studio provides API tokens for this; by default, requests may not require authorization. [28]

Context: how much text the model holds in front of it

Context is the volume of conversation and materials available to the model in the current request. It includes your instructions, the document, previous turns, and the response. It is limited, even if you could send messages all day long.

A value of 128K or 256K on a spec sheet is the model's maximum capability. On your PC, a large context will require additional memory. Tokens are small pieces of text that the program uses to measure this volume. To start, set 4096 tokens, then try 8192. For agentic coding work, Ollama recommends from 64K, but it must fit on your specific machine. [25]

If there are many documents, you can add RAG — searching for relevant snippets before answering. Then the model receives selected parts instead of the entire library. Check both the found snippets and the conclusion itself.

Code editor and agents

For the official Ollama integration with VS Code, you need to install the extension, run Ollama, and select the model in the editor chat. Requirements for the editor and program versions are provided in the instructions. [29]

For automatic work with files, you need a model that is good at using tools and has sufficient context. Test it first on a single error in a copy of the project. A small model might explain a function well but get lost during a long overhaul of the entire application.

# Linux/macOS shell; replace the path with your own: ./llama-server -m /path/to/model.gguf -c 4096 --host 127.0.0.1 --port 8080 # Windows PowerShell; replace the path with your own: .\llama-server.exe -m "C:\Models\model.gguf" -c 4096 --host 127.0.0.1 --port 8080 # GPU-backend depends on the downloaded build. # These are examples for different OSs, not commands for sequential execution.

To disable Ollama's cloud features, there is a parameter in the documentation OLLAMA_NO_CLOUD=1. After changing it, restart the program. In Windows, a variable in a new PowerShell will not affect an already running background Ollama — you need to change its environment and restart the application. [24]

How to understand if a model is right for you

Don't judge AI solely by a nice greeting. Give it a few of your usual tasks:

1. Shorten text. Were important facts preserved?
2. Answer based on a document. Is the answer in the provided material, or did the model make it up?
3. Explain or fix code. Does the result work after checking?
4. Work in Ukrainian. Are the language, terms, and tone okay?
5. Answer without the internet. Do you really need autonomous functions?

Compare two models on the same task. Note how long you waited and how much you had to fix. For your work, a fast model with an acceptable answer might be more useful than a larger one that makes you wait a long time.

The table can be scrolled sideways →

Frequently asked questions

Start small: install the program, download one model, and give it a task you would have done manually today. And if you need help, write your PC specs, the model name, and what isn't working in the comments. You can also discuss settings in our Ukrainian community.

Sources and clarifications

The material is based on software documentation and model cards. Memory tips are starting guidelines; speed and quality depend on your configuration. We have not conducted comparative tests of all models on a single PC. Below are links to check compatibility, licenses, and current terms.