Local AI vs cloud: comparing business costs

In a comparison of local and cloud AI, costs must be calculated for the same work outcome, rather than comparing a GPU with the price of a million tokens. The more economical option depends on volume, capacity, answer quality and operational responsibilities.

A downloadable model does not make the service free. Likewise, API charges do not necessarily represent the entire cloud expense. This article proposes a comparison method and a simulation using invented teaching figures, clearly separated from market prices. We are not presenting an economic return measured by Giallo Studio.

What is the right unit for comparing spending?

Choose an observable task: classifying a document, preparing a correct draft, extracting fields or answering with a verifiable source. Define what counts as completion and which human checks remain necessary.

Cost per technical request can be misleading. One user request may generate several calls, searches, attempts and answers requiring correction. The useful denominator is the task completed according to the agreed criteria.

For example, a cheap draft that needs almost complete rewriting is not equivalent to one usable after routine review. Before entering figures, use the local model evaluation protocol to ensure the two options genuinely perform comparable work.

Also define how to count an incomplete task. If an answer is interrupted, a document is not read or the user must start again, do not hide the event in the average of successful attempts. Keep the cost incurred and the reason for failure. This distinguishes a configuration that is economical because it is efficient from one that looks economical because the spreadsheet counts only successful attempts. The counting rule must be identical for local and cloud and written before collecting data.

Which items belong in the total cost of local deployment?

TCO, the total cost of ownership and management, includes more than the purchase. Consider setup, updates, support, energy, physical space, storage, backups and the tools needed to make the model available to people.

If you use an existing PC, the initial cost may differ from a new machine's. However, it is not reasonable to ignore alternative uses of the workstation, additional consumption or the administrator's time.

When sizing the system, distinguish RAM and VRAM on Windows from the Mac unified memory. A quotation that cannot support the required workload is not a valid economical alternative: it describes a configuration that does not complete the same work.

How do you assess cloud costs beyond a single rate?

Price lists may distinguish input, output, caching and tools. The official OpenAI API pricing and the Claude pricing documentation show these categories, which must be checked for the product and model actually used.

We do not quote current amounts: for an operational estimate, verify the official price list, currency, date and applicable conditions. Do not assume a personal chat subscription covers API integration.

Add the business application's expenses: access management, integration, monitoring, any storage and support. Some items apply to both options; treat them consistently rather than assigning them only to the option you want to lose.

Also check how the provider bills features other than text generation. Search, storage and other tools may use their own units or specific conditions. If the workflow does not use them, do not add them out of vague caution; if it does, do not omit them because the headline rate looks simpler. The cost record should connect each item to an observed part of the workflow and its contractual condition, distinguishing included features from usage-dependent charges.

How do you count tokens, attempts and documents?

Collect a representative sample and record the entire workflow. If a question requires document retrieval, do not count only the user's typed sentence: instructions, history and retrieved passages may enter the context.

The guide to calculating API costs helps organize input and output. On the local side, Ollama API metrics provide counts and timings useful for observing processing; they do not automatically measure the business's overall cost.

If a configuration requires more attempts, include them. If a person always corrects part of the work, record that time separately. Making these items visible shows whether a cost difference comes from model pricing or from how the process is designed.

How can you build a clear simulation?

Below we use teaching assumptions only, not quotations or average prices. We assume two systems complete the same work with acceptable quality and sufficient capacity; this equivalence must be established before using the calculation in a business.

For local deployment, we choose a 36-month management horizon. Spreading spending over this period is a comparison choice, not guidance on tax depreciation.

For cloud deployment, we assume 100 euros fixed per month plus 0.02 euros per completed task, with the variable charge including the attempts assumed in this simulation.

What does the example's break-even point tell us?

In the simulation, local deployment costs 300 euros per month within the assumed capacity. Cloud costs 100 + 0.02 × N, where N is the monthly number of completed tasks.

Break-even is reached at (300 − 100) / 0.02 = 10,000 tasks per month. This is an arithmetic consequence of the assumptions, not a market threshold.

The 15,000 row remains valid only if the machine supports that load without additional costs. No capacity has been measured here. The model excludes tax differences, financing and residual value: add them consistently if relevant to your comparison.

Why can changing one assumption reverse the result?

The calculation above is deliberately simple. In practice, support hours, document length, the effective rate and machine capacity can change.

If the example's variable cloud cost halves while everything else stays unchanged, break-even moves to 20,000 tasks. If instead monthly local costs rise to 400 euros and cloud remains at 100 + 0.02 × N, break-even becomes 15,000 tasks. These too are mathematical exercises, not forecasts.

Present at least a cautious scenario and a high-utilization scenario. The aim is not to pick favorable figures, but to understand which information is missing for a decision. If the result mainly depends on unmeasured administration hours, that is the first item to observe during the pilot.

How do you estimate energy without using the GPU's headline figure?

A component's stated consumption does not necessarily match the machine's consumption during work. A useful estimate needs observed average power, operating hours and the applicable energy cost.

The relationship is kWh = average kW × hours. For example, using purely hypothetical values, an average of 0.2 kW for 100 hours produces 20 kWh. At an invented tariff of 0.30 euros/kWh, that would cost 6 euros. This example is separate from the 10-euro allocation in the TCO table.

Also account for idle time, services left running and shared hardware. If you lack measurements, use a stated range and check its effect on the result. Do not turn a maximum specification or a review of another machine into your office's actual consumption.

How does quality enter the cost comparison?

A solution is cheaper only if it completes the required work within the necessary constraints. Define which errors prevent use and which can be corrected during normal review.

For data extraction, check fields and sources. For drafts, separate tone from factual correctness. For a RAG system for business documents, also evaluate retrieval and citations: the model can answer well using the wrong source.

If a configuration needs an additional check, include it. If both require the same review, do not charge it to only one. The cost comparison must reveal actual differences without assuming local and cloud quality is inherently equal or different.

Also assess the response to unavailability. Local deployment may require recovery or machine replacement; cloud deployment may require handling limits, connectivity or an external outage. You do not need arbitrary amounts for every risk: describe the planned procedure, responsible person and cost of a genuinely available alternative. If the service is essential, continuity costs deserve the same attention as model rates.

A distributor with recurring questions: when should local AI be considered?

Imagine a hypothetical scenario: a distributor frequently consults a relatively stable technical catalog. The workload is repetitive, and the business has someone who manages its infrastructure.

Local deployment can become a candidate if the model meets the task, capacity is adequate and administration fits the organization. The comparison must include catalog updates, access controls and support, not just answer generation.

First measure useful requests and peaks. A monthly volume spread evenly differs from the same volume concentrated during one shift. An apparently fixed cost may change if another machine or a queuing system is needed to meet service timings.

Do not promise savings before this check: document the option and the data that could support it.

A practice with irregular demand: when should cloud AI be assessed?

In a second hypothetical case, a practice uses the assistant mainly during certain periods and needs different capabilities for documents, writing and analysis.

Cloud deployment can be a candidate when it avoids unused capacity and allows service use without managing every component internally. Data-processing conditions, access, limits and supplier dependency still need assessment.

A hybrid design can keep some processing local and assign other work to external services. The Ollama cloud documentation reminds us that one ecosystem can offer different execution paths. The desktop application's name alone does not identify where data is processed.

Which mistakes make a quotation misleading?

The first is comparing the initial local purchase with a single month of cloud use. The second is ignoring the people administering the service. The third is using a cheaper model that does not complete the work while counting every call as a success.

Another mistake is promising continuous availability from a workstation that is switched off or used for other tasks. Finally, do not confuse permission to use the weights with free access to every tool or to support.

The Guide to local AI privacy completes the picture before confidential documents are introduced.

Which data should you collect before deciding?

Prepare a record covering tasks, volumes, peaks, minimum quality and the service owner. Include dated quotations and pilot measurements, clearly marking items still estimated.

Keep the initial outlay, recurring spending and management allocation separate: an option may have an attractive average cost but require cash or expertise the business prefers to use elsewhere. Reassess the comparison when workload or configuration changes.

To build a comparison using your process data, Giallo Studio's AI consultancy can define the pilot and cost model: the same task, verifiable results and explicit assumptions before choosing how to acquire or use capacity.