Running AI Applications on a VPS: What Hardware and Resources Do You Need?

The best VPS for AI depends on where the model runs – if your app uses a hosted model through an API, a server with 2 cores and 4 GB of RAM is often enough. On the other hand, running an open-source model on your own CPU takes more memory (around 8 GB for a 7B model or 16 GB for a 13B model), while training models and generating images are more demanding and typically require GPU hardware. Keep reading to learn everything you need to know about VPS for AI apps.

Key points:

• AI apps that call an external API need resources for the app stack, not for the model.
• Self-hosted models on CPU are limited by RAM size and memory bandwidth far more than by core count.
• 4-bit quantised models need roughly a third of the memory of the original 16-bit weights.
• Training, fine-tuning, image generation, and many concurrent users need a GPU.
• Even then, the app, database, and vector search can stay on a VPS.

What does running AI on a VPS actually involve?

AI on a VPS covers three very different workloads, and mixing them up is an easy way to end up with the wrong server. So, before comparing plans, work out which one you have:

The first two fit comfortably on a CPU-based AI VPS. The third doesn’t, and no amount of CPU cores will change that.

VPS for AI apps that call an API

A VPS for AI apps that rely on a hosted model needs about the same resources as any other web application. The model runs on the provider’s hardware, and your server receives requests, keeps conversation history, calls the API, stores results, and runs background jobs.

That covers a lot of real projects: a support chatbot on a company website, an n8n or Python automation that summarises incoming emails, a Discord or Telegram bot, a coding agent working on a repository around the clock. For these, 2 cores and 4 GB of RAM is a fine starting point. You can move to 8 GB once you add PostgreSQL with pgvector or a separate vector database such as Qdrant, plus a few worker processes.

However, what matters more than raw power here is that the server stays online, sits close to your users, and keeps your API keys on the server side instead of inside a browser or mobile app.

At MVPS, we offer one-click images for this kind of setup. The Claude VPS app installs the Claude Code CLI during provisioning, so you can go straight to authentication and your repositories, and the OpenClaw VPS app does the same for OpenClaw automations. For anything else, the Docker VPS image is a clean base for containerised AI stacks.

Can you run AI models on a VPS without a GPU?

The short answer is: yes. Small and mid-size open models run on a CPU-only VPS for AI models, as long as there’s enough RAM. Tools like Ollama and llama.cpp load quantised models in the GGUF format and run them on ordinary server processors.

Quantisation reduces memory usage by storing each model weight in 4 or 5 bits instead of 16, making the model roughly a third of its original size with only a modest loss in quality. Ollama’s default 4-bit models are a good starting point. 

According to Ollama’s documentation, you’ll need at least 8 GB of RAM for a 7B model, 16 GB for a 13B model, and 32 GB for a 33B model (1). Here’s what that means for actual model downloads:

The model file is only part of the picture: the context window also uses memory, and the longer the conversation, the more it needs. Once you add the operating system, your app, and a database, it’s worth leaving around 30% extra RAM beyond what the model itself requires.

Why memory bandwidth matters more than core count

When a model generates text, it reads essentially all of its weights from memory for every single token. That makes generation speed a memory bandwidth problem, not a CPU problem. 

A rough rule of thumb: tokens per second equals memory bandwidth divided by the size of the model in memory.

A real-world example from the llama.cpp project shows that a dense 32B model on a desktop with DDR5-5600 memory generated about 3.5 tokens per second, which matched what the bandwidth predicted (2). On a server where memory is shared between virtual machines, you should plan for numbers in that range or lower for models of that size.

This has practical consequences for anyone choosing a VPS server for AI:

• A smaller model answers faster, and halving the model’s size roughly doubles the generation speed.
• Extra cores help with reading long prompts, which is compute-heavy, but they barely move generation speed once memory is saturated.
• On a CPU, a 7–8B model is generally best suited to one user at a time. Smaller 2–4B models tend to respond faster, while models above 14B can feel sluggish.

Our advice: on CPU, pick the smallest model that passes your own quality test, then measure it on the actual server with llama-bench before building around it. It’s also worth checking that the virtual CPU exposes AVX2 instructions, since llama.cpp relies on them for speed.

Storage and the rest of the AI stack

NVMe storage matters mostly at load time – models are read from disk when the inference server starts, or switches models, so fast storage means shorter restarts. That’s why you should plan disk space generously: two or three 8B variants, an embedding model, Docker images, and a vector database can easily take 20–30 GB.

Embedding models are much lighter. For example, the popular nomic-embed-text model is just 274 MB (3) and runs well on a CPU, making it practical to build a local RAG pipeline that indexes your documents, even on a mid-range VPS. You can then use a hosted model to generate answers from those documents, keeping your document store on your own server while benefiting from a more powerful model.

When do you need a GPU instead of a VPS server for AI?

Some AI workloads simply don’t belong on a CPU-based VPS server for AI, and it’s better to know that before you pay. You need GPU hardware for:

• training or fine-tuning models, even with lightweight methods such as LoRA,
• image, video, or real-time speech generation at usable speeds,
• serving many users at once with quick responses,
• running models above roughly 30B parameters at interactive speed.

What works well is a split setup: the VPS hosts the application, database, vector search, authentication, and job queues, while the heavy generation goes to an API or a separate GPU machine.

You’ll also want to think about sustained workloads. Interactive AI requests usually come in bursts, which makes them a good fit for a VPS. Running a batch job that maxes out every CPU core for hours is a different story. If you need to process a large dataset in one go, a GPU or dedicated server is a better choice.

What is the best VPS for AI? A practical checklist

There’s no single best VPS for AI, but there is a clear set of things to check before you order:

• RAM for the model, the context window, and your app, with around 30% headroom.
• A modern CPU with AVX2 support.
• NVMe storage with room for several model files.
• A data centre close to your users, and inside the EU if prompts contain personal data.
• A fast upgrade path, so you can add RAM as your models grow.
• Root access, so you can run Docker, Ollama, or any other stack you choose.

What VPS for an AI model? Matching our plans to your workload

If you’re wondering what VPS for an AI model makes sense on our platform, this is where we’d start (prices are monthly and exclude VAT):

Not sure what to choose? Since upgrades take just 3–5 minutes and keep your IP address, you can start with a smaller plan and upgrade when your actual usage calls for it.

If your AI application needs more memory, our Performance VPS plans offer up to 128 GB of RAM. All plans come with KVM virtualisation, NVMe storage, and free backups across ten European locations. You can always compare your options in our online VPS configurator.

FAQ

Sources:

1. Ollama, README, GitHub repository ollama/ollama 
2. ggml-org/llama.cpp, GitHub issue #19480, CPU token generation and memory bandwidth measurements
3. Ollama Library, nomic-embed-text model page