Fix Image.reduce_colors/2 raising, corrupting output above 256 colors, and returning float images

[hlindset]

Fixes

Clamp :colors to the number of unique colors. The :colors option is now clamped to the number of unique colors in the image, instead of failing when it exceeded the image's pixel count. Matches what Image.Scholar.k_means/2 does for :num_clusters, and both paths now apply the bound through Image.Scholar.fit/3.

Silently wrong output above 256 colors. A cast to {:u, 8} wrapped every color index above 255 back around (256 became 0, 257 became 1, etc.). Pixels with a color index above 255 were painted with the color reached by the wraparound. The indices now stay in their original {:s, 32}.

Before fix (colors: 512), half the palette was unreachable:

After fix (colors: 512):

Raised instead of returning an error tuple. Image.Scholar.fit/3 did not translate the ArgumentError that Scholar.Cluster.KMeans.fit/2 raises for the checks it makes outside its option schema, nor the ArithmeticError it raises for a single sample (a 1px image).

Raised when the image could not be converted to a tensor. to_nx!/2 inside the body has been replaced by to_nx/2 in the existing with.

Added

Adds Image.Scholar.unique_color_count/1. Returns the number of distinct colors, taking either an image or its tensor. Image.Scholar.unique_colors/1 was the only way to get that number, and it does a lot of unnecessary work when all you need is the count.

Breaking

Returns {:u, 8} instead of {:f, 32}. K-means produced a float image, and nothing cast it back. The result is now rounded into the band format it clustered. libvips truncated these float values when casting on save, so output values can change by up to 1 per channel after this change.

Returns the image in the colorspace it was given. A :cmyk image returns four bands and a greyscale image returns one, where previously every image came back as 3-band :srgb. Clustering still happens in :srgb and the result is converted back, so :cmyk output is slightly lossy (though not visible in practice), and a 16-bit image comes back 16-bit with its colors drawn from an 8-bit palette.

[kipcole9]

This really great. Just wondering if, instead of casting back to {:u, 8}, the cast should be back to the format of the original image?

[hlindset]

This really great. Just wondering if, instead of casting back to {:u, 8}, the cast should be back to the format of the original image?

The round + {:u, 8} cast is only to get it back to integer values, as the data inside is still just fractional within an 8-bit (sRGB) range. But I agree, it's probably not expected behaviour to pass in CMYK and get back sRGB. Since we run clustering in sRGB, some accuracy is lost in the roundtrip for CMYK, and for 16-bit formats in the initial sRGB conversion.

Might be worth mentioning in the docs that if the source image is CMYK, and the user wants sRGB output at the end, it's better to convert to sRGB once before reducing colors, because CMYK degrades on each conversion, e.g.:

iex(1)> px = Image.new!(1, 1, color: [200, 40, 0, 30], interpretation: :cmyk); Image.get_pixel(px, 0, 0)
{:ok, [200, 40, 0, 30]}
iex(2)> px = Image.to_colorspace!(px, :srgb); Image.get_pixel(px, 0, 0)
{:ok, [0, 138, 190]}
iex(3)> px = Image.to_colorspace!(px, :cmyk); Image.get_pixel(px, 0, 0)
{:ok, [194, 44, 2, 30]}
iex(4)> px = Image.to_colorspace!(px, :srgb); Image.get_pixel(px, 0, 0)
{:ok, [1, 137, 187]}
iex(5)> px = Image.to_colorspace!(px, :cmyk); Image.get_pixel(px, 0, 0)
{:ok, [194, 43, 3, 34]}

For other interpretations it's a one time cost.

1. I'll add a to_colorspace at the end to get it back to the original colorspace instead of returning sRGB.
2. I can explore running the clustering in Lab space. Should produce better colors and have virtually no 16-bit precision loss. Best left for another PR though.

[kipcole9]

Yes, you're right, clustering in RGB isn't a great strategy for colour fidelity. LAB - or maybe okLAB - would be a better choice by far.

[kipcole9]

And thinking further about dominant_color/1 as well, should also be done in LAB or okLAB and in that case probably should be only clustering on the ab components, not L.

[hlindset]

Updated with conversion back to the image's original colorspace