Download Ollama or LM Studio, choose a model, and you can launch a local chatbot within minutes. The service requires no subscription fees and can operate without an internet connection, ensuring your data remains on your computer. However, there is a limitation: the models you can run and their response speed are entirely dependent on your computer's performance. A typical computer with 16GB of RAM can run relatively smaller models with around 7 to 8 billion parameters, such as Meta's Llama 3.1 8B or Alibaba's Qwen3 8B, and a graphics card with 8GB of VRAM can run these models more quickly.
However, these models still fall significantly short compared to the cutting-edge models behind today's ChatGPT or Claude. If you wish to run models that are closer in capability, you will need at least 32GB of RAM and a graphics card with 16GB or more of VRAM, which is not cheap at current prices.
Requirements for Running Local Chatbot Models
The memory required by a model is almost linearly proportional to its size, measured in billions of parameters (B). Most people run compressed ("quantized") versions of local models, where each parameter is stored using about four bits, reducing the model size to about a quarter of its original size. In Ollama's library, Llama 3.1 8B has a size of 4.9GB, Microsoft's Phi-4 is 9.1GB, Google's Gemma 3 27B is 17GB, while Meta's Llama 3.3 70B is 43GB.
Ollama's general guideline is that a 7B model requires 8GB of RAM, a 13B model requires 16GB, and a 33B model requires 32GB. OpenAI's gpt-oss-20b has 21 billion parameters, but only uses 360 million parameters per token, achieved through a design known as "mixture of experts." This allows for quick responses, but the model still needs to fit within the available memory. OpenAI states it can "run with just 16GB of memory," but in Ollama it requires 14GB, nearly exhausting all the memory of a 16GB computer.
Relationship Between Speed and Memory Bandwidth
Each time a model generates a word, or more accurately, a token, it must retrieve all parameters one by one. Thus, speed primarily depends on memory bandwidth rather than raw processing power. Nvidia engineers describe this stage as "memory-bound." The speed of a graphics card's VRAM is several times faster than the RAM of the rest of the computer, so models that can run on the GPU operate much faster than those that spill over into system memory. As a rule of thumb, a speed of generating five to ten tokens per second can keep pace with most people's reading speed.
In tests by LMSYS, Nvidia's RTX 5090 was found to generate text from OpenAI's gpt-oss-20b at a rate of 205 tokens per second. In contrast, benchmarks from llama.cpp showed that Apple's base M4 chip could generate about 24 tokens per second on a 7B model.
Ollama tested Nvidia's DGX Spark, an AI development computer priced at US$3,999 (approximately HK$31,199), which generated text at a rate of 38 tokens per second on Llama 3.1 8B, while on the 70B version, it was only 4.42. I personally ran Mistral 7B on a 2024 Asus Zenbook Duo equipped with 32GB of RAM and an Intel AI Boost NPU.
Through Ollama, Mistral pushed CPU usage to 52%, with memory usage around 17GB. This level of usage is manageable, but if other resource-intensive applications are running simultaneously, it can slow down the laptop. Using Intel's OpenVINO toolkit, the model maintained NPU usage at 98%, while CPU usage was only 3%, and the laptop remained responsive while generating about 2,500 words (nonsensical content).
In the April 2026 rankings by Artificial Analysis, Alibaba's Qwen3.5 27B scored 42 in its intelligence index, on par with OpenAI's GPT-5 at medium settings, while Google's Gemma 4 31B scored 39. However, leading cutting-edge cloud models scored as high as 57, and Qwen3.5 27B requires 17GB to 20GB of memory, which clearly exceeds the capacity of a 16GB computer.
Yet even the most powerful computers can barely meet the demands of cutting-edge intelligence. Moonshot's Kimi K2.6 is a model with 1 trillion parameters, launched in April 2026 as one of the best open-weight models, requiring approximately 600GB of RAM and VRAM combined, according to Unsloth. The company released a compressed version of the open model. Its successor, Kimi K3, scored 57 in the artificial analysis intelligence index in July 2026, just behind Anthropic's Claude Fable 5 and OpenAI's GPT-5.6 Sol.
However, this model has 2.8 trillion parameters, with weights amounting to about 1.4TB, TECHi estimates, making it nearly impossible to run on an eight GPU data center server. Therefore, for occasional private summaries or some offline programming assistance, a 16GB computer is sufficient. To utilize features comparable to alternative cloud chatbots, 32GB of RAM or a 16GB graphics card is required. According to Tom's Hardware's RAM price index, a 32GB DDR5 kit alone costs US$434 (approximately HK$3,385), nearly equivalent to the monthly US$20 (approximately HK$156) subscription for ChatGPT Plus or Claude Pro.
Subscription costs would amount to nearly two years of expenses. However, any near-cutting-edge demands still rely on data centers.
Item Specifications RAM 16GB / 32GB Graphics Card VRAM 8GB / 16GB Model Size 4.9GB (Llama 3.1 8B) Generation Speed 38 tokens per second (DGX Spark) Price US$3,999 (approximately HK$31,199)

