[SDXL] The True Value of the Refiner! What Emerges from Destructive Creation

2024-10-232025-2-13
Four-frame manga of a spaceship colliding with an asteroid and leaking air English.png (1133×1600)
  • The Refiner enhances texture.
  • The Refiner disrupts structure.
  • Restoring the image through high-resolution rendering reveals its full potential.

Introduction

Hello, it's Easygoing here.

Today, let's take a deep dive into Stable Diffusion XL's Refiner feature!

It All Started with the Refiner

When I first got into image-generating AI, the Refiner feature was what caught my attention.

[SDXL] What is the Refiner? Do You Really Need It? | AI image journey

In SDXL, using the Refiner model to fine-tune illustrations created with the Base model enhances their overall quality.

The Refiner has a unique effect, bringing individuality to the images, but it also tends to distort their structure, making it a challenging feature to master.

Anime illustration of a female operator nervous about an emergency on a spaceship 3 Refiner hires.png (2048×2048)

After about four months of using the Refiner, I’ve gained some insights, and today, I’d like to share them with you.

Let's Compare the Results!

First, let’s compare illustrations with and without the Refiner.

We’ll start by generating an image at SDXL's default resolution.

Anime illustration of a female operator nervous about an emergency on a spaceship 5.png (1024×1024)
Original 1024 x 1024

A beautiful illustration of a female operator has been generated.

The Refiner Destroys Structure!

Next, let’s apply the Refiner to this illustration.

Anime illustration of a female operator nervous about an emergency on a spaceship 5 Refiner.png (1024×1024)
Refiner 1024 x 1024

As mentioned earlier, the Refiner significantly disrupts the structure. In this case, the contours are distorted, and parts like the hands are disfigured.

The Refiner is recommended to be used as a final touch after the Base model has completed the image. However, because it distorts the structure so much, you can't apply it too heavily.

What Changed with the Refiner?

So, what exactly changed when we applied the Refiner?

Let’s first compare the brightness between the original and refined images.

Anime illustration of a female operator nervous about an emergency on a spaceship 5 histogram.png (1052×552)
Original
Anime illustration of a female operator nervous about an emergency on a spaceship 5 Refiner histogram.png (1056×552)
Refiner

Compared to the original, the refined image has reduced mid-tones, with more emphasis on the light and dark extremes. This shows that the contrast has been enhanced, giving the image a more striking appearance.

If we look closely, we can see that it's not just the contrast that has increased. Shadows on the face and hair are more pronounced, adding depth, and the colors are more vibrant.

High Resolution Brings Out Its True Potential!

Next, let’s try increasing the resolution of the image.

Anime illustration of a female operator nervous about an emergency on a spaceship 5 Hires.png (2048×2048)
Original 2048 x 2048
Anime illustration of a female operator nervous about an emergency on a spaceship 5 Refiner Hiress.png (2048×2048)
Refiner 2048 x 2048

Both images look good at higher resolution, but the refined version stands out. The structure is restored after re-rendering at high resolution.

By incorporating the Refiner in between the process rather than at the end, it can have a more profound effect and reveal its true value.

Highlights Are Enhanced!

The effects of the Refiner show up in unexpected places as well.

Anime illustration of a female operator nervous about an emergency on a spaceship 4 Hires.png (2048×2048)
Original 2048 x 2048
Anime illustration of a female operator nervous about an emergency on a spaceship 4 Refiner Hires.png (2048×2048)
Refiner 2048x 2048

Using the Refiner increases contrast, which emphasizes the highlights in the eyes. Additionally, by distorting the structure, the highlights are slightly shifted.

AI is skilled at accurately depicting subjects, but it's not as good at creating intentional distortions. By applying the Refiner, you introduce subtle shifts that give the illustration a more hand-drawn warmth.

More Comparisons

Let’s showcase some more examples. The top image is the high-resolution original, and the bottom is the version refined with the Refiner.

Woman in a Blue Dress

Anime illustration of a woman in a blue dress smiling at us at a party 2 Hires.png (1664×2368)
Anime illustration of a woman in a blue dress smiling at us at a party 2 Hires Refiner.png (1664×2368)
Overall weight and texture of the dress

Autumn Couple

Anime illustration of a young couple amidst the lights of autumn leaves Hires.png (2048×2048)
Anime illustration of a young couple amidst the lights of autumn leaves Refiner Hires.png (2048×2048)
Enhanced color of the faces and leaves

The Refiner naturally boosts contrast and saturation while keeping the softness of the characters intact. It adds texture while preserving the illustration's gentle feel.

The Challenge of Using the Refiner

The Refiner is fascinating, but it’s quite challenging to use.

Anime illustration of a female operator nervous about an emergency on a spaceship 2 Refiner Hires.png (2048×2048)

The effect of the Refiner varies greatly depending on the base illustration, and achieving the right balance without ruining the image requires multiple trial and error attempts.

It’s ideal for when you’re taking your time on a single piece, but it’s not very practical for quickly generating multiple images.

Why Implement It as a Standard Feature?

Given its difficulty, why was the Refiner included as a standard feature in SDXL?

This is just my opinion, but when SDXL was released, there was a perception that Midjourney was the go-to for artistic expression, while Stable Diffusion was known for realistic rendering.

Anime illustration of a female operator nervous about an emergency on a spaceship 6 Refiner Hires.png (2048×2048)

At the time, Stability AI (the company behind Stable Diffusion) was taking an aggressive approach to development, and I believe the Refiner was one way they aimed to address artistic expression with Stable Diffusion XL.

By making such a complex feature standard, Stability AI showed a bold and innovative spirit.

Lessons Learned from the Refiner

Personally, I’ve learned a lot from using the Refiner.

  • Combining multiple models to enhance results.
  • Breaking down an illustration and then rebuilding it.

These concepts seem simple in hindsight, but they weren’t obvious at first.

Anime illustration of a female operator smiling and waving as she turns toward us Refiner Hires.png (2048×2048)

The lessons from the Refiner form the foundation of my current workflow, where I use SDXL and Flux.1 together.

While the Refiner didn’t become widely popular, I believe it has influenced the aesthetic approach of newer generative AIs, like Flux.1.

Conclusion

  • The Refiner enhances texture.
  • The Refiner disrupts structure.
  • Restoring the image through high-resolution rendering reveals its full potential.

Generative AI is in its infancy, and new ideas are constantly emerging.

Anime illustration of the blue earth seen from space.png (2048×2048)

This excitement is something only those willing to dive into this unfinished territory can experience. It's a special privilege we have right now.

Why not take this opportunity to explore AI yourself?

Thank you for reading until the end!