What Is Inpainting? Masked AI Photo Edits, Explained

Inpainting is an AI edit that redraws only the region of a photo you paint over and leaves every pixel outside it exactly as it was. (That painted area is called a mask.) You paint over the part you want changed: the cluttered background, the stranger in the corner, a jacket you want in a different color. The model then redraws only that region, guided by your prompt. That is the difference from AI edits that redo the whole photo and risk changing everything.

The word comes from art restoration. There, an inpainter fills the damaged patches of a canvas so they match the work around them. The AI version does the same job with an image-generating model (a diffusion model). The difficulty is keeping the new paint consistent with the old. It has to match light, perspective, and texture across a boundary the viewer should never notice.

Photo editors have had healing and clone tools for decades. Those tools can only copy pixels that already exist somewhere in the image. Diffusion inpainting can make things that were never there at all: a different garment, an open sky, an object lit to match the scene. That is a different capability, with different failure modes.

The mask: a stencil for the model

A mask is a black-and-white overlay the same shape as your image. The areas you paint mean “redraw this.” Everything else means “hands off.” In an app you draw it with your finger. Inside the app it becomes a score for every pixel, and each stage of the edit checks that score.

Your prompt then describes what should appear in the painted region, in context. As with any diffusion edit, describe the result you want, not the instruction. To remove a car from a driveway, paint over the car and prompt for what takes its place: “empty concrete driveway, fallen leaves.” The model always fills the hole with something. Your prompt decides what.

How the region blends instead of showing seams

The obvious approach is to make a patch on its own and paste it in. That leaves obvious seams. The lighting does not match, and the texture stops dead at the boundary. Diffusion inpainting avoids this with two mechanisms.

Per-step re-anchoring. A diffusion model builds an image by cleaning noise away over several passes, as covered in our plain-language explainer on diffusion models. Each pass is called a denoising step. During inpainting, the app does one extra thing after every step. It paints your original image back over the whole unmasked area, adding exactly as much noise to that original as the current step expects to see. The model stays free to invent inside the mask, but at every step it is working against the true surroundings rather than its own drift. Because the model always sees the whole frame, the fill inherits the scene’s lighting direction, perspective, and grain instead of arriving from nowhere.

Feathered compositing. At the end, the new region is blended back into the untouched original. First the edges of the mask are softened over a band of pixels. That softening is called feathering. Instead of a hard on-or-off boundary, the transition ramps gradually from fully new to fully original. Small mismatches in color or texture end up smeared across that band, where your eye cannot lock onto them.

Mask hygiene: the difference between okay and invisible

Most bad inpainting results are mask problems. Two habits fix the majority of them:

• Mask generously. Trace an object too tightly and its edge pixels survive outside the mask. That means shadows, stray hair, and the soft halo where the object met the background. Those leftovers show up as a ghost outline around the thing you “removed.” Push the mask well past the object’s outline, and cover its shadow and reflections too. The model is good at inventing background. It cannot fix pixels you told it to preserve.
• Let the edges stay soft. The softened band needs room to work. Sharp, jagged mask edges give the blend nowhere to ramp. A crisp mask boundary can also print itself into the result as a faint contour. A soft brush and margins measured in tens of pixels beat pixel-perfect tracing.

One more habit: prompt for the fill. The area outside the mask is already settled. Spend your words describing what belongs inside the mask and how it is lit. “A wooden bench, dappled shade, matching the park behind it” gives the model more to work with than restating the entire scene. Naming the surrounding light helps too, with words like “overcast” or “warm evening sun from the right.” It is the cheapest way to make the fill match its neighbors.

Expect to try more than once. Generation is seeded randomness, so the first fill is one draw from a range of results the model could have given you. A second attempt with the same mask often lands better purely by luck.

What inpainting is good for, and where it struggles

The classic uses map to three verbs:

• Replace a background: mask everything behind the subject and describe a new setting. The subject keeps its own pixels, so you get no drifted face and no melted hands.
• Remove an object: mask it generously, then prompt for the background that should have been there. The best results come on textured, repeating surroundings: grass, sky, walls, and water.
• Change one element: mask a jacket and prompt “red wool coat.” Mask a wall and repaint it. The shape stays, the material changes.

Inpainting has limits. It invents: removing a person gives you a plausible background, not the true one. Very large masks turn the job into text-to-image with a border, which loses the benefit of context. Structured, unique backgrounds are hard to continue convincingly, such as lettering or a specific bookshelf. And some edits have to connect distant parts of the image, like moving a light source or changing the time of day. Those call for a redraw of the full frame instead of a mask. That technique is img2img.

Inpainting in Onhand, on the device

Onhand’s masked inpainting on iPhone workflow uses Turbo. Paint the region to change, describe the desired result, and generate. The pipeline re-anchors the unmasked latent region during denoising and composites the result with feathered edges. The boundary is a blend, so inspect it closely when exact detail matters.

Masked inpainting is not offered by every model in the library. Keep a copy of the original, work on a small region, and compare the output at full size. The edit runs locally; exported images follow the settings of Photos or the sharing destination you choose.

PhoneDiffusion also lists inpainting; Onhand vs PhoneDiffusion compares the two apps’ editing tools side by side.