[SDXL] What is the Refiner? Do You Really Need It?

2024-9-302025-2-13

Refiner article thumbnail.png (1000×1000)

Refiner improves image quality
Using too much Refiner can distort images
making it unnecessary for regular use.

Introduction

Hello, Easygoing here.

Today, I’ll be discussing a question that came to mind when I first started using SDXL. This post will feature romantic images of couples, focusing on facial expressions and skin textures.

While there’s a bit of personal preference in the subject matter, the main aim is to showcase the characters' emotions and textures.

Refiner’s Introduction

When SDXL was released in July 2023, one of the key features that caught attention was the implementation of the Refiner.

AI image generation gradually creates images from noise, with the Base model handling the core aspects of the image, and the Refiner model refining the details, resulting in higher image quality.

How SDXL Works

SDXL pipeline.png (1837×647) — [stabilityai/stable-diffusion-xl-base-1.0 · Hugging Face](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0)

In SDXL, you can specify the point at which the process switches from the Base model to the Refiner model.

Settings screen for the Refiner in webUI Forge.png (1843×725) — Example from Stable Diffusion webUI Forge

In the diagram above, it's set to 0.8, meaning the Base model handles the first 80% of the process, and the Refiner takes over for the remaining 20%.

Do You Really Need the Refiner?

When I researched the Refiner, there wasn’t much information available.

Most articles I found suggested that the Refiner's impact was minimal, and using it only slowed down generation without providing significant benefits.

So, I decided to investigate the Refiner's effect myself.

Since I could only find the official base model for the Refiner and there weren't many customization examples, I tested the effect using the default base model.

Drawing with the Refiner

I started by creating images with the Refiner.

It seems that the Refiner isn’t specifically designed for drawing, as only about 10% of the generated images turned out well, with many featuring deformed faces or hands.

However, when successful, the images had a unique, simple charm.

Illustration 1 drawn with the Refiner.png (512×768)

Illustration 2 drawn with the Refiner.png (512×768)

Foreground and Background

One clear feature of images generated with the Refiner is the distinct separation between the characters (foreground) and the background.

Characters (Foreground)

Clear outlines
Limited color palette: typically three shades—highlight, mid-tone, and shadow
Thin boundary lines

Background (Distant View)

Blurred overall
Slightly darker tones?

Just as our brains focus on what we’re looking at and blur out everything else, the Refiner seems to apply a similar effect.

What’s the Best Ratio?

Next, I explored the optimal ratio between the Base and Refiner models.

The Refiner struggles with structure, often producing distorted, almost horror-like images at the beginning.

Base 0%, Refiner 100%

BASE0% Illustration of a young couple on a skyscraper rooftop, ruined due to drawing failure.png (1240×1240)

Base 20%, Refiner 80%

BASE20% Illustration of a young couple on a skyscraper rooftop, similarly ruined.png (1240×1240)

Base 40%, Refiner 60%

BASE40% Illustration of a young couple on a skyscraper rooftop, still ruined.png (1240×1240)

Base 60%, Refiner 40%

BASE60% Illustration of a young couple on a skyscraper rooftop, with an incomplete, anime-like result.png (1240×1240)

Base 80%, Refiner 20%

BASE80% Illustration of a young couple on a skyscraper rooftop, significantly better.png (1240×1240)

Base 100%, Refiner 0%

BASE100% Illustration of a young couple on a skyscraper rooftop, clean and polished.png (1240×1240)

Best After 80%

It seems that the best results come when the Base model handles at least 80% of the process.

The Base model I used, anima_Pencil-XL_v500, is of such high quality that using it at 100% works perfectly fine.

One thing to note is that the Refiner isn’t great at drawing, so you’ll need to increase the steps for the Base model to finish most of the image.

Instead of switching halfway through, think of adding the Refiner as a final touch after the image is complete.

Next, I’ll try gradually increasing the influence of the Refiner.

100% to 80%

Base 100%, Refiner 0%

BASE100% Illustration of a young couple on a skyscraper rooftop, cleanSkin looks smooth.png (1240×1240) — Skin looks smooth

Base 95%, Refiner 5%

BASE95% Illustration of a young couple on a skyscraper rooftop, cleanSkin looks slightly softer.png (1240×1240) — Skin looks slightly softer

Base 90%, Refiner 10%

BASE90% Illustration of a young couple on a skyscraper rooftop, cleanMore depth and dimension.png (1240×1240) — More depth and dimension

Base 85%, Refiner 15%

BASE85% Illustration of a young couple on a skyscraper rooftop, cleanBackground gains some weight.png (1240×1240) — Background gains some weight

Base 80%, Refiner 20%

BASE80% Illustration of a young couple on a skyscraper rooftop, cleanFacial expressions start to blur.png (1240×1240) — Facial expressions start to blur

This part is where preferences might differ.

The anima_Pencil-XL_v500 model excels at skin texture. Using the Refiner adds a bit of moisture, but if overused, the skin becomes overly smooth, like that of an android.

When it comes to facial expressions, the Refiner is a downside—overdoing it results in a loss of subtlety.

For backgrounds, a light touch of the Refiner adds a nice watercolor-like depth, so perhaps the model was trained with watercolors in mind.

Personally, I think the best advantage of using the Refiner is that it can separate characters from the background without thickening the lines.

Conclusion: You Don’t Really Need It

Refiner improves image quality
Using too much Refiner can distort images
making it unnecessary for regular use.

There are several drawbacks to using the Refiner. The biggest issue is that it disrupts the delicate balance that the model creators have worked hard to achieve.

It also slows down the image generation process, so it’s not really necessary for everyday use.