All articles

Deep dives

What On-Device AI Costs in Battery and Heat

Image generation is close to the hardest work a phone can do. What that means for battery drain, thermal throttling, and keeping generations fast.

By · · Updated · 7 min read

Making an image on your phone costs battery and makes heat. This is close to the heaviest non-stop work you can ask a phone to do. An image model (a diffusion model) holds the graphics chip (GPU) near full effort for seconds on end. That is the one load shape phone hardware is least built for.

The phone warms up. The battery meter drops faster than usual. After several images in a row, each one arrives more slowly than the first did. Heat is expected under sustained load, though software efficiency and model choice can change how much work is needed.

Why image generation is an unusual load for a phone

The number that matters here is the power level a chip can hold indefinitely without getting too hot. That ceiling is set by physics. The body is sealed. There is no fan; heat must move through the device’s passive cooling structure and enclosure. The battery sits millimetres from the part making the heat.

Phones are engineered around short bursts followed by idle time, because that is what nearly all use looks like. Those idle gaps are when built-up heat gets shed. Making an image removes them. Gaming is the closest everyday comparison, and even a demanding game leaves more slack than a denoising loop does.

Where the energy goes

Nearly all of it goes into the middle of the job. A generation has three parts. First the model reads your prompt. Then it cleans up a field of noise a little at a time, over a series of passes. (Each pass is a denoising step.) Last it turns that compressed result (the latent) into pixels. The middle part dominates, because every step is a full run through the largest network in the chain. Two steps cost roughly twice what one step costs. The prompt reader (the text encoder) and the final converter (the decoder) are rounding errors beside that.

encodedenoising steps (repeated)full network pass, every stepdecodeenergy per generation scales with step count
The shape of one generation. A small text-encode block at the start and a small decode block at the end sit either side of a loop that runs the biggest network once per step. Not to scale. The point is that the repeated middle block is where the energy goes.

Inside a step, arithmetic is not the only cost. Moving bytes between memory and the graphics chip burns power too. On modern chips that traffic often costs about as much as the multiplications it feeds. An image model holds billions of numbers (weights), and all of them have to stream through the compute units on every single step. Last step’s traffic cannot be reused. So a smaller model is cheaper for a reason beyond fitting in memory. Fewer bytes moved per step means less energy burned per step. That is why step count is the setting that matters most and why few-step turbo models are the practical answer on a phone.

Thermal throttling, and why the fourth image is slower

When the device gets too warm, the system slows its own chip down on purpose. That protects the battery and the components. (The technical name is thermal throttling.) It is the designed behaviour. The practical effect is that a run of generations gets slower as it goes. The first one starts from a cool device and gets full clock speed. After enough sustained work, the chip may run at a reduced rate; the number of images varies. It is shedding heat more slowly than the denoising loop is producing it.

The slowdown takes a few generations to appear because of thermal mass. The chip heats quickly. The frame and glass around it take time to reach the temperature where the system steps in. iOS reports its reading of that state to apps, so an app can tell it is being throttled. Knowing does not make the heat go away, and no software trick buys back the lost clock speed.

Why the same prompt is not always the same speed

Conditions outside the model change how fast a generation finishes, and the app does not put any of them on screen. A warm device has less thermal headroom left to spend. A low battery makes the system draw less power. Low Power Mode caps performance outright, and a long, steady load on the graphics chip is the first thing to hit that cap.

  • Let it cool. A minute of idle between heavy bursts wins back more speed than pushing on through the slowdown does.
  • Unplug for long sessions. Charging makes its own heat inside the same case. Stack that on a sustained load on the graphics chip and the phone hits the throttling point sooner than either would alone.
  • Use the model that fits the session. Begin with Turbo and a lower quality choice for exploration. Compare other models when their look is useful. A smaller download or shorter wait does not, by itself, establish lower energy per image.

The cloud relocates this cost

A cloud generator shifts the main computation to server hardware. That reduces the generation workload on your phone, although its screen and network still consume power. Comparing total energy requires measurements of the specific model, hardware, and service; it cannot be inferred from where the computer sits.

Running locally puts the energy cost on the device you are holding, where it competes with the rest of your day. In exchange, your prompts and photos never leave the device, generation works with no connection, and there is no charge per image. Whether that trade lands in your favour depends on what you value. Both sides are laid out in cloud versus on-device compared.

What this means for Onhand

Onhand runs image generation locally, using MLX on the GPU or a Core ML route depending on the model. Turbo offers short step schedules for quick exploration; the other models have their own quality and memory trade-offs. Loading and decoding also consume time and energy.

Use a short session on your own device to judge the wait and warmth. Pause when the device gets hot, lower quality if the result still serves your purpose, and use Stop when you no longer need a run. Stop takes effect at a safe operation boundary; it cannot forcibly interrupt an in-flight GPU operation. We do not have a measured battery percentage per image to quote.

Cloud services such as Midjourney move this cost onto their own servers and charge a subscription for it; Onhand vs Midjourney compares that trade with an on-device app. For what to expect from Onhand in speed, see the offline AI image generator for iPhone.

Sources

Primary sources for the claims above.