All articles

Explainers

AI Image Generation: A Plain-Language Glossary

Latents, VAEs, guidance, steps, seeds, quantization and the rest: short definitions of the terms you meet in on-device image generation.

By · · Updated · 9 min read

Most people meet AI image generation through a text box and a button. Behind that screen sits a pile of jargon. Some of it is necessary: you cannot work out why a slider behaves the way it does without knowing what a latent is. The rest came out of research papers and was never put into ordinary words. This glossary translates it.

The terms are grouped by where you run into them. First comes the loop that turns noise into a picture. Then the parts of the model doing the work, and the controls an app puts in front of you. After that, the words for editing a photo you already have. Last, the words that only matter once the model runs on a phone instead of a server. Where a term needs more than two sentences, the definition links to a full article on it.

The generation loop

The model does not draw your picture in one go. It starts with a field of random noise, which looks like television static, and removes a little on each pass. Your description steers every pass. What is left at the end is the image. These are the terms for that loop.

  • Diffusion. The training trick behind all of this. Take real images and add noise to them, a bit at a time, until nothing is left but static. Then teach a network to undo one bit of that damage. Start the trained network on pure noise instead, and it paints rather than repairs. The idea is unpacked in diffusion models, explained simply.
  • Denoising step. One pass of the network over the image in progress, leaving it a little cleaner. Each step is the same fixed amount of arithmetic. That is why step count changes less than people expect.
  • Scheduler (or sampler). The bookkeeping that decides how much noise to remove on each step, and how far apart the steps sit. Different schedulers take different routes to a finished image. A model tuned for few steps expects a particular one.
  • Latent. The small grid of numbers the model cleans up. Roughly 64×64 numbers stand in for a 512×512 picture. The grid holds shape and color relationships rather than pixels. Working there instead of on full pixels is what makes the loop affordable.
  • Latent space. The map those grids live on. The model learned it during training, so points that sit near each other decode into pictures that look alike. That is why a small nudge to a latent gives you a sensible variation rather than garbage.

The parts of a model

A diffusion model is three networks that work together. One reads your prompt. One does the repeated cleanup. One turns the result into a picture you can look at. Downloads are large because you are getting all three.

  • Text encoder. The piece that turns your prompt into numbers the denoiser can work from. It learned from image–caption pairs, so it understands descriptions of pictures specifically. That is why unusual phrasing can land badly.
  • U-Net (or denoiser). The biggest network of the three, and the one that runs on every step. It reads the current noisy latent along with your encoded prompt. Then it predicts the noise to subtract.
  • VAE. A pair of small networks. One squeezes a picture down into a latent, the other expands a latent back into pixels. (The full name is variational autoencoder.) Every generation ends with a VAE decode, and every photo you feed in starts with a VAE encode.
  • Parameters. The numbers the networks learned during training, roughly a billion of them for a compact model. Multiply that count by the bytes each number takes and you have close to the file size; the text encoder, the VAE, and the packaging around them add a little on top. That is why these models are gigabytes large.
  • Checkpoint. A saved snapshot of all those numbers. It is the file people mean when they say they downloaded a model.

The controls you touch

Most apps give you four or five knobs. Each one acts on a specific part of the loop above. Once you know which part, you can adjust it on purpose. Otherwise you are turning it up because higher sounds better.

  • Seed. The number that decides the starting noise. Keep the seed, the prompt, and the settings the same, and you get repeatable results when the full pipeline is deterministic. That makes it the tool for repeating a result, or for changing one thing at a time. It is covered in why the same prompt gives a different image.
  • Guidance (CFG). The setting that makes your prompt bite. On each step the model guesses twice: once with your words in mind, once ignoring them completely. It then pushes the result away from the guess that ignored you. The scale sets how hard it pushes. Low is loose and natural. High is literal, and eventually over-saturated. Guessing twice also doubles the cost of every step. (The technical name is classifier-free guidance.)
  • Negative prompt. A second description, this time of what you do not want. It works only as the thing guidance pushes away from. Switch guidance off and the field does nothing at all. It is explained in why negative prompts do nothing in turbo models.
  • Distillation. Training a small, fast student model to copy a slow teacher model. The student reaches in one or two steps what the teacher takes fifty steps to reach.
  • Turbo. The name given to models trained that way. They finish in one to four steps and usually run without guidance. That is what makes them workable on a phone; see SDXL Turbo explained. The trade is less fine control and slightly less detail.

Editing an image you already have

Editing runs the same loop. The only difference is where it starts. Instead of pure noise, it begins with your photo with some noise mixed in. That one change is the mechanism behind every AI edit feature, and the terms below are variations on it.

  • Img2img. Squeeze an existing image down to a latent, add noise to it, then clean it back up with a new prompt. The result keeps the original composition while taking on the new description. It is the basis of AI photo editing without uploading anything.
  • Strength. How much noise goes in before the cleanup restarts, given as a fraction. Low strength leaves the photo mostly intact. High strength wipes it toward a fresh generation. It is the one slider that controls an AI photo edit.
  • Inpainting. Img2img held to one region. The model redraws inside the area you select. The rest of the photo is kept and pasted back around it. More in what is inpainting.
  • Mask. The black-and-white map that marks which pixels are allowed to change. Its edges are usually feathered, meaning blurred over a few pixels. The blur lets the new region blend in instead of showing a hard seam.
  • Outpainting. The same idea aimed outward. You extend a picture past its original frame by inpainting into empty canvas. It is described in what is outpainting.

Terms that only matter on a phone

On a phone the problem stops being speed and becomes fit. iOS caps how much memory a single app may hold, and it shuts down any app that goes over. There is no swap file to fall back on. So most on-device vocabulary is about squeezing a multi-gigabyte model under a fixed ceiling.

  • Quantization. Storing each learned number with fewer bits, 4 instead of 16 for example. That cuts model size by about 75% with modest quality loss. How it works and where it breaks is the single biggest reason these models fit at all.
  • Unified memory. Apple silicon gives the CPU and GPU one shared pool of RAM instead of separate banks. Model weights never have to be copied between them. Graphics, the system, and your model all draw on the same budget. See what the chip in your iPhone provides.
  • Staged loading. Bringing each piece into memory only while it is needed. Run the text encoder, let it go, then load the denoiser. Peak memory use then matches the largest single piece rather than all of them added together.
  • Tiled decoding. Turning the final latent into pixels in overlapping patches instead of all at once. A full-frame VAE decode causes a memory spike far larger than its short runtime suggests. Tiling flattens that spike. Both techniques appear in what running Stable Diffusion on an iPhone really takes, and RAM, not storage, decides which models fit.
  • On-device, local, offline. People swap these three words freely, but they say different things. On-device means the computation happens on your own hardware. Local makes the same claim about where processing runs. Other app services can still transmit data. Offline is the test you can run yourself, because it still works in airplane mode. An app can be marketed as private and be none of the three. That is what makes the airplane-mode test useful.

Where these words show up in Onhand

Onhand uses MLX and Core ML to run prepared image models on compatible iPhones and iPads. Turbo is one option in a broader library that includes Detailed Images, Detailed Scenes, Realistic Photos, Art & Fantasy, and Anime Illustration. Quality choices map to different step counts by model. Guidance and negative-prompt settings are internal; the app does not expose a seed entry field or a seed-lock control.

Image generation, photo editing, and Refine run locally. Masked inpainting uses Turbo. Download and try the model before working offline. Catalog updates, downloads, purchases, and limited analytics use network services; see the privacy policy.

For how these terms play out with no connection, see the offline AI image generator for iPhone. If you would rather adjust many of these settings yourself, Onhand vs Draw Things compares Onhand with an app that exposes far more of them.

Sources

Primary sources for the claims above, so you can check them rather than take our word for it.