On-device AI changes where the bill lands. With Apple Intelligence and Gemini Nano the model runs on the user’s phone or laptop, so there is no per-request cloud fee, but the costs move into engineering time, device coverage limits and narrower model capability. ROI comes from lower inference spend, better privacy positioning and faster features, measured against a cloud model baseline.
What counts as on-device AI today?
Apple Intelligence is Apple’s set of system features and developer frameworks that run a compact language model locally on supported iPhone, iPad and Mac hardware, with Private Cloud Compute used for heavier tasks. Developers can call the on-device model through Apple’s Foundation Models framework. Gemini Nano is Google’s smallest Gemini model, built into Android through AICore on supported devices and exposed to web developers through experimental built-in AI APIs in Chrome.
Both share the same trade. The model is small enough to fit in device memory, so it handles summarising, rewriting, classification and short extraction well. It is not a replacement for a large cloud model on long reasoning, broad world knowledge or large documents.
Where do the costs actually sit?
Teams used to cloud APIs often assume on-device means free. It is free per call, but the project is not. The budget shifts into four places.
- Engineering: integrating platform frameworks, handling availability checks, writing fallbacks and testing across device generations.
- Coverage gaps: only recent, higher-end devices support these models, so a share of your audience still needs a cloud path or no feature at all.
- Quality work: prompt design and evaluation for a smaller model take more iterations, because it fails differently from a large one.
- Maintenance: operating system updates can change model behaviour, which means re-testing on each major release.
The cloud alternative has the opposite shape: low setup, ongoing per-token fees that scale with usage. That difference is the core of the ROI calculation.
How do you budget an on-device AI feature?
Break the budget into a one-off build and a recurring run cost. The build is mostly developer weeks. The run cost is close to zero for supported devices and equal to your cloud fallback cost for the rest.
A worked framing
For example, imagine a retail app that adds a “summarise reviews” button. On supported devices the summary is generated locally. On older devices it calls a cloud model. Your monthly run cost is then the number of summaries requested on unsupported devices multiplied by the cloud price per summary. As the device base upgrades, that number shrinks without any new work.
- Estimate monthly feature usage from analytics on similar features.
- Split that usage by supported versus unsupported devices using your own audience data.
- Price the unsupported share at your cloud provider’s current rate.
- Add the build cost, amortised over the expected life of the feature.
This gives a break-even point: the month in which cumulative cloud savings exceed the extra build effort.
What returns are realistic to claim?
The cleanest return is avoided inference spend. If a high-volume feature moves from cloud to device for most users, the saving is visible on the invoice. It only matters, though, when volume is high. A feature used a few hundred times a day rarely justifies a separate on-device path.
The second return is privacy. Processing on the device means personal text, photos or messages do not leave it, which simplifies some compliance conversations under rules such as GDPR and Turkey’s KVKK. That is hard to put a number on, but it can unblock features a legal team would otherwise refuse.
The third return is speed and offline use. Local inference avoids a network round trip, and features keep working on a train or in a store with poor signal. Measure it as latency and completion rate, not as a vague benefit.
Which KPIs prove the value?
Pick a short list and track it from launch. Too many metrics hide the signal.
- Share of feature requests served on-device versus cloud fallback.
- Cloud inference spend per month for the feature, before and after.
- Median response time on-device compared with the cloud path.
- Feature adoption and repeat use among supported users.
- Task success rate from user feedback or a small manual review sample.
If adoption is low, cost savings are irrelevant. Treat usage as the first gate, then look at money.
How Alien Road applies on-device AI in optimisation projects
A typical engagement begins with an audit of the client’s existing AI features and their cloud bills. The team flags tasks that are short, frequent and privacy-sensitive, because those are the best candidates for local models. Long-form generation and research tasks stay in the cloud.
During setup the team works alongside the client’s developers to define fallbacks and an evaluation set: a few hundred real, anonymised inputs with agreed good outputs. Both the on-device and cloud paths are scored against the same set, so quality is compared on evidence rather than impressions.
Measurement runs through the client’s analytics stack, usually Firebase or a product analytics tool, with events for which path served each request. The monthly report shows device coverage, spend saved, latency and quality scores side by side, plus a recommendation on whether to extend on-device processing to the next feature or keep it where it is.
When is on-device AI the wrong choice?
Some situations argue against it. Being clear about them is part of an honest ROI case.
- Low volume features, where cloud fees are already trivial.
- Audiences on older or budget hardware, where coverage would be small for years.
- Tasks that need current information or large context windows.
- Marketing websites without an app, where browser support for built-in models is still experimental.
In these cases a well-chosen cloud model with caching is usually cheaper overall.
Where to start this week
Export last month’s AI-related cloud invoice and list the features behind it. For each one, note the request volume and how long the typical input is. Short, frequent, private tasks go on a shortlist.
Then check your analytics for the share of users on devices that support Apple Intelligence or Gemini Nano. Multiply the two and you have a first estimate of how much spend an on-device path could move. If the number is small, stop there. If it is large, a two-week prototype with an evaluation set is the next sensible step.
Keep reading: AI Optimization · AI Optimization