Small language models are AI models with far fewer parameters than frontier systems, typically from under a billion to a few tens of billions, that can run cheaply in the cloud or on a single machine. In marketing work they are best at narrow, repeated tasks such as classifying search terms, tagging content, cleaning data and routing messages, where speed, cost and data control matter more than broad reasoning.
Well-known families include Microsoft’s Phi models, Google’s Gemma, Meta’s smaller Llama variants, Mistral’s compact models and Alibaba’s Qwen line, along with the “mini”, “flash” or “haiku” tiers that major providers offer through their APIs. Many open-weight models can be downloaded and run locally with tools such as Ollama, llama.cpp or LM Studio, so data never leaves the company’s own hardware.
What is the difference between a small and a large model?
Size affects what a model knows and how well it reasons across many steps. A large frontier model can draft a strategy, follow long instructions and handle unusual requests. A small model is narrower: it handles a clear, well-defined task quickly, but it is more likely to stumble on ambiguous instructions or questions outside its training.
The practical differences are cost and speed. A small model can process thousands of short items in the time a large model handles a fraction of them, at a much lower price per item. When the job is “label these 50,000 search terms into twelve intent groups”, that difference decides whether the project is affordable.
Which marketing tasks suit small models?
The common thread is high volume, short inputs and a fixed set of possible outputs.
- Search term classification: sorting Google Ads search terms into intent groups or negative keyword candidates.
- Content tagging: assigning categories, topics or product types to thousands of pages or posts.
- Sentiment and topic labelling for reviews, social comments and survey responses.
- Data cleanup: normalising company names, job titles or addresses in CRM exports.
- Routing: deciding which team should handle an incoming form or message.
- Language detection and simple translation checks across multilingual content.
Tasks that need judgment, creativity or long documents usually stay with larger models or people.
Why does running a model locally matter?
Some data should not go to a third-party API at all: customer lists, unreleased campaign plans, health or financial details. A small open-weight model running on the client’s own server or a private cloud instance keeps that data in place, which simplifies compliance under GDPR and Turkey’s KVKK.
Local models also give predictable costs. Once the hardware exists, processing another hundred thousand rows costs electricity rather than API fees. The trade-off is maintenance: someone has to update, monitor and secure the setup.
How Alien Road applies small language models
In a typical engagement our AI optimisation team uses small models as workers inside larger pipelines rather than as chat tools. Discovery starts with the task list: we look for repeated classification or cleanup jobs that people currently do in spreadsheets, and estimate the volume per month.
Build a labelled sample first
Before choosing a model, we have the client’s specialists label a few hundred real examples, such as search terms or reviews, with the categories they actually use. That sample becomes the test set for every model we try.
Compare models on that sample
We run two or three candidates, usually a small open-weight model, a small API model and a large model as a reference, against the labelled sample. If the small model agrees with the human labels nearly as often as the large one, it wins on cost and speed. If not, we adjust the prompt, add examples, or keep the large model for that task.
Put people at the edges
Low-confidence items go to a human review queue. For ad accounts, the model suggests negative keywords; a specialist approves them before they reach Google Ads or Microsoft Advertising.
For measurement we track agreement with human labels over time, review queue size, processing cost per thousand items and the hours freed. Monthly reports note any drift, for example when a new product line produces search terms the model misclassifies.
What are the limits and risks?
Small models make different mistakes from large ones, and they make them more often on edge cases.
- They follow format instructions less reliably; validate outputs against the allowed labels in code.
- Knowledge is thinner, so they may not recognise niche brand or product names without examples.
- Performance in Turkish and other non-English languages varies by model; test with real local data.
- Licences differ between open-weight models, and some restrict commercial use; check before deploying.
- A local setup still needs security patches and access control like any server.
How do you decide between fine-tuning and prompting?
Start with prompting and a handful of examples in the prompt. It is cheap to change and often good enough. Fine-tuning a small model on a few thousand labelled examples can raise accuracy on a narrow task, but it adds training work and a model version to maintain. We consider it only when prompting stalls below the required agreement level and the task volume justifies the effort.
What does a small model cost to run?
Costs depend on the route. Through an API, small model tiers are priced per token and are usually a fraction of the large tier; check current price pages, since they change often. Running locally, a recent laptop with enough memory can handle models of a few billion parameters for testing, while production volumes usually justify a dedicated server or a cloud GPU instance. The hidden cost is people: setup, monitoring and periodic re-testing against the labelled sample.
A simple checklist before you start
- Define the task as a closed set of outputs, not an open question.
- Collect and label at least 200 to 300 real examples.
- Decide which data may leave your systems and which may not.
- Test at least one small and one large model on the same sample.
- Set a confidence threshold and a human review path for items below it.
- Log inputs and outputs so errors can be traced.
Where to start this week
Export last month’s search terms report from one Google Ads account. Take a random 300 rows and label each one yourself with three or four intent groups, plus “irrelevant”. Then run the same rows through a small model with a short prompt listing those groups, and count where it agrees with you. That one afternoon tells you whether automating the weekly search term review is realistic.
Keep reading: AI Optimization · AI Optimization