Skip to content

Fix Image.reduce_colors/2 raising, corrupting output above 256 colors, and returning float images #229

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hlindset merged 2 commits into from
Aug 7, 2026
Merged

Fix Image.reduce_colors/2 raising, corrupting output above 256 colors, and returning float images #229

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c487e14
Fix Image.reduce_colors/2 contract, >256-color corruption, band format
hlindset Aug 6, 2026
64d17d1
Return the reduced image in the colorspace it was given
hlindset Aug 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,12 @@

* **Breaking:** `Image.k_means/2` and `Image.reduce_colors/2` return `{:error, %Image.Error{reason: :invalid_option}}` for an invalid or unknown option instead of raising `NimbleOptions.ValidationError`. `Image.k_means!/2` and `Image.reduce_colors!/2` raise `Image.Error` rather than the NimbleOptions exception. `operation` is set to `k_means` or `reduce_colors`, and `value` is `{key, value}` for an invalid value or the list of keys for unknown options. ([#227](https://github.com/elixir-image/image/pull/227))

* **Breaking:** `Image.reduce_colors/2` returns `{:u, 8}` instead of `{:f, 32}`. K-means produced a float image, and nothing cast it back. The result is now rounded into the band format it clustered. libvips truncated these float values when casting on save, so output values can change by up to 1 per channel after this change. ([#229](https://github.com/elixir-image/image/pull/229))

* **Breaking:** `Image.reduce_colors/2` returns the image in the colorspace it was given. A `:cmyk` image returns four bands and a greyscale image returns one, where previously every image came back as 3-band `:srgb`. ([#229](https://github.com/elixir-image/image/pull/229))

* `Image.reduce_colors/2` clamps `:colors` to the number of unique colors in the image. Previously a `:colors` greater than the image's pixel count failed. ([#229](https://github.com/elixir-image/image/pull/229))

* `Image.affine/3` and `Image.rotate/3` now premultiply alpha explicitly only when the background is non-opaque, since libvips handles the other cases itself. `Image.shear/4` and `Image.translate/4` inherit this. ([#217](https://github.com/elixir-image/image/pull/217))

### Fixed
Expand All @@ -48,6 +54,12 @@

* Fix the `:greater_than` and `:less_than` documentation for `Image.chroma_mask/2`, `Image.chroma_key/2` and `Image.replace_color/2`, which described the bounds the wrong way round in all six places they appeared. The mask covers the range between the two, so `:greater_than` is the lower bound and `:less_than` the upper. ([#224](https://github.com/elixir-image/image/pull/224))

* Fix `Image.reduce_colors/2` silently producing wrong output above 256 colors. A cast to `{:u, 8}` wrapped every color index above 255 back around (256 became 0, 257 became 1, and so on), so those pixels were painted with the color reached by the wraparound. ([#229](https://github.com/elixir-image/image/pull/229))

* Fix `Image.reduce_colors/2` raising instead of returning an error tuple, both for the `ArgumentError` that `Scholar.Cluster.KMeans.fit/2` raises for checks it makes outside its option schema, and for the `ArithmeticError` it raises when given a single sample. ([#229](https://github.com/elixir-image/image/pull/229))

* Fix `Image.reduce_colors/2` raising when the image could not be converted to a tensor. ([#229](https://github.com/elixir-image/image/pull/229))

### Removed

* **Breaking:** Removes `Image.Options.WarpPerspective`, replaced by `Image.Options.Mapim`. ([#216](https://github.com/elixir-image/image/pull/216))
Expand Down Expand Up @@ -108,6 +120,8 @@ This is the changelog for Image version 0.70.0 released on July 8th, 2026. For

* Adds `Image.YUV.valid_encodings/0` and `Image.YUV.valid_colorspaces/0`.

* Adds `Image.Scholar.unique_color_count/1`, which returns the number of distinct colors in an image and accepts either an image or its tensor. ([#229](https://github.com/elixir-image/image/pull/229))

### Changed

* `Image.average/1` now weights the average by the alpha band so transparent pixels do not contribute; fully transparent images fall back to the unweighted color-band average, and float-format images return unrounded averages.
Expand Down
85 changes: 53 additions & 32 deletions lib/image.ex
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9066,9 +9066,9 @@ defmodule Image do
@doc """
Reduces the number of colors in an image.

Takes the `k_means/2` of the image and then
re-colors the image using the returned cluster
colors.
Applies K-means clustering to the pixels of the image
and then re-colors each pixel with the color of the
cluster it was assigned to.

### Arguments

Expand All @@ -9080,6 +9080,9 @@ defmodule Image do

* `:colors` is the number of distinct colors to be
used in the returned image. The default is `#{@default_clusters}`.
An image cannot have more distinct colors than it started
with, so `:colors` is clamped to the number of unique colors
in the image.

* See also `Scholar.Cluster.KMeans.fit/2` for the
available options.
Expand All @@ -9089,6 +9092,11 @@ defmodule Image do
* Note the performance considerations described in
`Image.k_means/2` since they also apply to this function.

* Clustering is performed in the `:srgb` colorspace and the
result is converted back to the colorspace of `image`. The
colors of a 16-bit image are therefore drawn from an 8-bit
palette.

* If the intent is to reduce colors in order to
reduce the size of an image file it is strongly advised to
use the appropriate arguments when calling `Image.write/2`.
Expand All @@ -9111,6 +9119,9 @@ defmodule Image do
"""
@doc subject: "Clusters", since: "0.50.0"

@spec reduce_colors(image :: Vimage.t(), options :: Keyword.t()) ::
{:ok, Vimage.t()} | {:error, error()}

def reduce_colors(%Vimage{} = image, options \\ []) do
case do_reduce_colors(image, options) do
{:error, reason} -> {:error, Image.Error.wrap(reason, operation: :reduce_colors)}
Expand All @@ -9119,42 +9130,41 @@ defmodule Image do
end

defp do_reduce_colors(image, options) do
with {:ok, image} <- to_colorspace(image, :srgb) do
kmeans_num_clusters =
Keyword.get(options, :colors, @default_clusters)

options =
options
|> Keyword.put(:num_clusters, kmeans_num_clusters)
|> Keyword.delete(:colors)

{width, height, bands} =
Image.shape(image)

nx_reshaped =
image
|> to_nx!()
|> Nx.reshape({height * width, bands})

with {:ok, model} <- Image.Scholar.fit(nx_reshaped, options) do
indicies =
Nx.as_type(model.labels, :u8)

model.clusters
|> Nx.take(indicies)
|> Nx.reshape({height, width, bands})
|> Image.from_nx()
end
options =
options
|> Keyword.put(:num_clusters, Keyword.get(options, :colors, @default_clusters))
|> Keyword.delete(:colors)

colorspace = Image.colorspace(image)

with {:ok, srgb} <- to_colorspace(image, :srgb),
{:ok, tensor} <- to_nx(srgb),
{width, height, bands} = Image.shape(srgb),
nx_reshaped = Nx.reshape(tensor, {height * width, bands}),
{:ok, unique_count} <- Image.Scholar.unique_color_count(tensor),
{:ok, model} <- Image.Scholar.fit(nx_reshaped, options, unique_count),
# The clusters are floats, so the recolored image is rounded
# back into the band format of the image being clustered.
{:ok, reduced} <-
model.clusters
|> Nx.take(model.labels)
|> Nx.round()
|> Nx.as_type(Image.band_format(srgb))
|> Nx.reshape({height, width, bands})
|> Image.from_nx() do
# Clustering happens in :srgb, so the result is returned to the
# colorspace the caller passed in.
to_colorspace(reduced, colorspace)
end
end

@doc """
Reduces the number of colors in an image or
raises an exception.

Takes the `k_means/2` of the image and then
re-colors the image using the returned cluster
colors.
Applies K-means clustering to the pixels of the image
and then re-colors each pixel with the color of the
cluster it was assigned to.

### Arguments

Expand All @@ -9166,6 +9176,9 @@ defmodule Image do

* `:colors` is the number of distinct colors to be
used in the returned image. The default is `#{@default_clusters}`.
An image cannot have more distinct colors than it started
with, so `:colors` is clamped to the number of unique colors
in the image.

* See also `Scholar.Cluster.KMeans.fit/2` for the
available options.
Expand All @@ -9175,6 +9188,11 @@ defmodule Image do
* Note the performance considerations described in
`Image.k_means/2` since they also apply to this function.

* Clustering is performed in the `:srgb` colorspace and the
result is converted back to the colorspace of `image`. The
colors of a 16-bit image are therefore drawn from an 8-bit
palette.

* If the intent is to reduce colors in order to
reduce the size of an image file it is strongly advised to
use the appropriate arguments when calling `Image.write/2`.
Expand All @@ -9197,6 +9215,9 @@ defmodule Image do
"""
@doc subject: "Clusters", since: "0.51.0"

@spec reduce_colors!(image :: Vimage.t(), options :: Keyword.t()) ::
Vimage.t() | no_return()

def reduce_colors!(%Vimage{} = image, options \\ []) do
case reduce_colors(image, options) do
{:ok, image} -> image
Expand Down
140 changes: 120 additions & 20 deletions lib/image/scholar.ex
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@ if match?({:module, _module}, Code.ensure_compiled(Scholar.Cluster.KMeans)) and
[Scholar](https://hex.pm/packages/scholar) machine-learning
primitives.

The primary public API is `unique_colors/1` and `k_means/2`
which underpin `Image.k_means/2` and `Image.reduce_colors/2`.
The public API is `unique_colors/1`, `unique_color_count/1` and
`k_means/2`, which underpin `Image.k_means/2` and
`Image.reduce_colors/2`.

"""

Expand Down Expand Up @@ -102,20 +103,130 @@ if match?({:module, _module}, Code.ensure_compiled(Scholar.Cluster.KMeans)) and
end
end

@doc """
Returns the number of unique colors in an image.

Prefer this over `unique_colors/1` when only the count is
needed.

### Arguments

* `image_or_tensor` is any 3- or 4-band `t:Vix.Vips.Image.t/0` with
`{:u, 8}` band format, or the `{height, width, bands}` tensor of
such an image as returned by `Image.to_nx/2`. Pass the tensor when
one is already to hand, to avoid converting the image twice.

### Returns

* `{:ok, count}` or

* `{:error, reason}`.

### Example

iex> Image.Scholar.unique_color_count(Image.new!(4, 4, color: :red))
{:ok, 1}

"""
@spec unique_color_count(image_or_tensor :: Vimage.t() | Nx.Tensor.t()) ::
{:ok, non_neg_integer()} | {:error, Image.Error.t()}

def unique_color_count(image_or_tensor)

def unique_color_count(%Vimage{} = image) do
with {:ok, tensor} <- Image.to_nx(image) do
unique_color_count(tensor)
end
end

# The rank is checked first so the band lookup below cannot raise.
def unique_color_count(%Nx.Tensor{} = tensor) do
cond do
Nx.rank(tensor) != 3 ->
{:error,
scholar_error(
"unique_color_count requires a {height, width, bands} tensor. " <>
"Found rank #{Nx.rank(tensor)}"
)}

Nx.axis_size(tensor, 2) not in [3, 4] ->
{:error,
scholar_error(
"unique_color_count requires a 3- or 4-band image. " <>
"Found #{Nx.axis_size(tensor, 2)} bands"
)}

Nx.type(tensor) != {:u, 8} ->
{:error,
scholar_error(
"unique_color_count requires an 8-bit unsigned image. " <>
"Found #{inspect(Nx.type(tensor))}"
)}

true ->
{:ok, do_unique_color_count(tensor, Nx.axis_size(tensor, 2))}
end
end

defp do_unique_color_count(tensor, bands) do
encoded =
tensor
|> encode_colors(bands)
|> Nx.flatten()
|> Nx.sort()

# Nx.diff/1 needs at least two elements, so a single pixel is counted
# directly. Otherwise the distinct count is one more than the number
# of adjacent unequal pairs.
if Nx.size(encoded) < 2 do
Nx.size(encoded)
else
Nx.to_number(Nx.sum(Nx.not_equal(diff(encoded), 0))) + 1
end
end

defp scholar_error(message) do
%Image.Error{message: message, reason: message}
end

# Scholar.Cluster.KMeans.fit/2 validates its options with
# NimbleOptions.validate!/2 which raises on an invalid or unknown
# option, so translate the exception to {:error, %Image.Error{}} here
# at the boundary
# Scholar.Cluster.KMeans.fit/2 raises rather than returning an error:
# NimbleOptions.ValidationError for an invalid or unknown option, and
# ArgumentError for the checks it makes outside its schema. Both are
# translated to {:error, %Image.Error{}} here at the boundary.
#
# `max_clusters` bounds `:num_clusters`, which cannot exceed the
# number of distinct samples to cluster.
@doc false
def fit(samples, options) do
{:ok, Scholar.Cluster.KMeans.fit(samples, options)}
def fit(samples, options, max_clusters \\ nil) do
# Scholar raises ArithmeticError for a lone sample rather than
# validating it, so it is checked here to keep the message useful.
if Nx.axis_size(samples, 0) < 2 do
{:error,
scholar_error(
"K-means requires at least 2 samples to cluster. " <>
"Found #{Nx.axis_size(samples, 0)}"
)}
else
{:ok, Scholar.Cluster.KMeans.fit(samples, clamp_clusters(options, max_clusters))}
end
rescue
exception in NimbleOptions.ValidationError ->
{:error, invalid_option(exception)}

exception in ArgumentError ->
{:error, %Image.Error{reason: :invalid_option, message: Exception.message(exception)}}
end

defp clamp_clusters(options, nil), do: options

defp clamp_clusters(options, max_clusters) do
case Keyword.fetch(options, :num_clusters) do
{:ok, num_clusters} when is_integer(num_clusters) ->
Keyword.put(options, :num_clusters, Kernel.min(num_clusters, max_clusters))

_other ->
options
end
end

# An unknown option sets :key to the list of unknown keys and leaves
Expand Down Expand Up @@ -159,25 +270,14 @@ if match?({:module, _module}, Code.ensure_compiled(Scholar.Cluster.KMeans)) and
"""
def k_means(%Vimage{} = image, options \\ []) do
with {:ok, {_count, colors}} <- unique_colors(image) do
# K-means requires at least as many samples as clusters, so
# the cluster count is clamped to the number of unique colors.
# A single unique color (solid image) is duplicated because
# the random centroid initialisation needs at least 2 samples.
unique_count = Nx.axis_size(colors, 0)

colors =
if unique_count == 1, do: Nx.concatenate([colors, colors]), else: colors

options =
case Keyword.fetch(options, :num_clusters) do
{:ok, num_clusters} when is_integer(num_clusters) ->
Keyword.put(options, :num_clusters, Kernel.min(num_clusters, unique_count))

_other ->
options
end

fit(colors, options)
fit(colors, options, unique_count)
end
end

Expand Down
Loading