Canvas cache performance regression with large images in 0.14

[dirkhillbrecht]

Is your issue REALLY a bug?

• My issue is indeed a bug!
• I am not crazy! I will not fill out this form just to ask a question or request a feature. Pinky promise.

Is there an existing issue for this?

• I have searched the existing issues.

Is this issue related to iced?

• My hardware is compatible and my graphics drivers are up-to-date.

What happened?

I draw an image from a source of arbitrary size and pixel source into a Canvas which may have a different size. The Canvas uses a Cache so the actual drawing code is in the Cache's closure parameter.
On updates, the image is flickering and for prolonged times replaced with white content. This is the more severe the larger the original image is - the size of the Canvas does not really matter.
The attached demo program shows the problem on my platform (Linux, X11, Ubuntu 24.04, Rust 1.92). Start it with the side length of a blue image rectangle and resize the screen. With a size length of 100 ("cargo run 100") rescaling is smooth. From size ~700 onward, the redrawing becomes more and more flickerish.

// Minimal demo program for the Iced 0.14 flicker behaviour

// Situation is: An image is generated from a pixel source and then drawn in a Canvas
// The Canvas uses a Cache internally so the actual drawing code resides in the closure parameter of the Cache.
// The drawn image is emulated by a blue square, in practice this could be a way more complex image.

// Use: "cargo run <pixel size>" and then rescale the frame
// "cargo run 100" → resizing is smooth, blue rectangle is shown all time
// "cargo run 900" → resizing becomes sluggish, white images appear during resizing
// "cargo run 2000" → slugginess increases vastly

// Cargo.toml:
/*
# Minimum Cargo file

[package]
name = "iced-canvas-flicker-repro"
version = "0.1.0"
edition = "2021"

[dependencies]
iced = { version = "0.14.0", features = ["image", "canvas"] } # GUI library
*/

use iced::widget::canvas::{self, Cache, Geometry};
use iced::widget::{canvas::Canvas, container};
use iced::{Element, Length, Rectangle, Size, Task, Theme};
use std::env;

fn main() -> iced::Result {
    // Get image size from command line argument
    let args: Vec<String> = env::args().collect();
    let size = if args.len() > 1 {
        args[1].parse::<u32>().unwrap_or(900)
    } else {
        900
    };

    println!("Creating {}x{} pixel image", size, size);
    println!("Try resizing the window to see the issue");

    iced::application(
        move || AppState::new(size),
        AppState::update,
        AppState::view,
    )
    .title("Canvas Flicker Reproduction")
    .run()
}

struct AppState {
    image_size: u32,
}

impl AppState {
    fn new(size: u32) -> (Self, Task<Message>) {
        (Self { image_size: size }, Task::none())
    }

    fn update(&mut self, _message: Message) -> Task<Message> {
        Task::none()
    }

    fn view(&self) -> Element<'_, Message> {
        let canvas = Canvas::new(BlueCanvas {
            image_size: self.image_size,
        })
        .width(Length::Fill)
        .height(Length::Fill);

        container(canvas)
            .width(Length::Fill)
            .height(Length::Fill)
            .into()
    }
}

#[derive(Debug, Clone)]
enum Message {}

struct BlueCanvas {
    image_size: u32,
}

impl canvas::Program<Message> for BlueCanvas {
    type State = CanvasState;

    fn draw(
        &self,
        state: &Self::State,
        renderer: &iced::Renderer,
        _theme: &Theme,
        bounds: Rectangle,
        _cursor: iced::mouse::Cursor,
    ) -> Vec<Geometry> {
        let start = std::time::Instant::now();

        let geometry = state.cache.draw(renderer, bounds.size(), |frame| {
            println!(
                "[DRAW] Cache rebuild starting for {}x{}",
                self.image_size, self.image_size
            );

            // Create blue pixels
            let pixel_count = (self.image_size * self.image_size) as usize;
            let mut pixels = vec![0u8; pixel_count * 4]; // RGBA

            for i in 0..pixel_count {
                pixels[i * 4] = 0; // R
                pixels[i * 4 + 1] = 0; // G
                pixels[i * 4 + 2] = 255; // B (blue)
                pixels[i * 4 + 3] = 255; // A (opaque)
            }

            // Create image handle
            let image = canvas::Image::new(iced::widget::image::Handle::from_rgba(
                self.image_size,
                self.image_size,
                pixels,
            ))
            .filter_method(iced::widget::image::FilterMethod::Linear);

            // Draw image to fill the frame
            let image_size = Size::new(self.image_size as f32, self.image_size as f32);
            let frame_size = frame.size();

            // Scale to fit
            let scale =
                (frame_size.width / image_size.width).min(frame_size.height / image_size.height);

            let scaled_size = Size::new(image_size.width * scale, image_size.height * scale);

            let position = iced::Point::new(
                (frame_size.width - scaled_size.width) / 2.0,
                (frame_size.height - scaled_size.height) / 2.0,
            );

            let bounds = Rectangle::new(position, scaled_size);

            frame.draw_image(bounds, image);

            println!("[DRAW] Cache rebuild finished in {:?}", start.elapsed());
        });

        println!("[DRAW] Total draw call took {:?}", start.elapsed());

        vec![geometry]
    }
}

struct CanvasState {
    cache: Cache,
}

impl Default for CanvasState {
    fn default() -> Self {
        Self {
            cache: Cache::new(),
        }
    }
}

What is the expected behavior?

No flickering during redraw for any image size as it was in 0.13.1.

Version

crates.io release

Operating System

Linux

Do you have any log output?

[edwloef]

Image handles should be stored in app state instead of being recreated on every view or draw, the flickering is because images are now loaded and decoded asynchronously instead of blocking the renderer.

[dirkhillbrecht]

Thanks for the reply! I already thought about this. However, my real-world use case needs the Canvas size to create the image and I see no way to obtain this information outside the draw() method. Am I missing something?

[dtzxporter]

Thanks for the reply! I already thought about this. However, my real-world use case needs the Canvas size to create the image and I see no way to obtain this information outside the draw() method. Am I missing something?

Implement the update method, and prepare and store the image there instead. It also provides the bounds!

[dirkhillbrecht]

You mean the Canvas' update() method, @dtzxporter , don't you? But how do I get an event delivered there to recompute the image data? I understand that Canvas' update() method only gets events originating from interaction with canvas. And: I cannot access the program's state from there, only the state of the Canvas. That does not come together to me.
And the program's update() function isn't working for this either, as it does not get the bounds of the Canvas (and that makes actually sense…).
To me, the logic to perform this computation makes still totally sense in the Cache's draw() method. Only there all the needed information comes together. If the cache would simply retain the outdated information until the (asynchronously executed) drawing closure returns, the problem would go away.

[dtzxporter]

The program update method has the same information as render, minus the renderer, which includes the bounds:

https://docs.rs/iced/latest/iced/widget/canvas/trait.Program.html#method.update

[dirkhillbrecht]

Ok, for me to get this clear:

Current architecture

• app's state contains canvas' cache
• If app detects canvas' content to be outdated: app's update() calls cache.clear()
• canvas' draw() "sees" invalidated cache, closure is called, cache content is rebuilt during draw()

Problem: If closure takes too long to complete, interface flickers as it presents cleared "empty" cache in the mean time.

Proposed architecture

• app's state contains some kind of flag "canvas must be redrawn" which might be set to "true" somehow (this must be more elaborate if redraw requirements are scheduled too fast, but that is left as an exercise…)
• cache is moved into canvas' state (which actually sounds much cleaner now I write it down)
• canvas' state also contains the "Image" used to fill the canvas
• canvas' update() listens for "RedrawRequested" window event.
• If that event occurs AND said flag is set, update() performs recomputation of the Image
• After Image is recomputed, canvas' update() calls Cache.clear() for the cache in the canvas' state.
• canvas' draw() works with Cache as now, but the closure - more or less - only contains the frame.draw_image() call.

Do I understand this right, @dtzxporter ? Sorry if this all sounds trivial, but I am still learning Rust and Iced and the whole architecture - while being really sophisticated and powerful - still seems to be a bit alien to a guy who did his last deep GUI development with Java and Swing back in the early 2000s…

[edwloef]

If closure takes too long to complete

This isn't quite the issue. Creating an image handle now doesn't actually decode and upload the image: that happens at some point in the future, ideally in time for the next frame, but iced no longer blocks the entire app on that happening. This is how I would go about doing this:

• app state holds the image handle, and loans it to the canvas
• the canvas's state holds its last known bounds
• when the RedrawRequest event happens you compare the current bounds to the old ones, if they've changed you send a message to the app
• your app creates the new image::Handle when it receives that message, starts an image::allocate task, and possibly aborts an already running image::allocate task via its task::Handle
• when that allocation comes back, swap the old image for the new one, since it's ready to be drawn

[dirkhillbrecht]

Thanks, @edwloef , this makes things even clearer to me. Having the Canvas to message something to the app is an established concept, so this sounds sensible. I think, I will sleep over this…

[yunfengwangluo]

Thanks, @edwloef , this makes things even clearer to me. Having the Canvas to message something to the app is an established concept, so this sounds sensible. I think, I will sleep over this…谢谢，这让我更清楚了。通过Canvas向应用发送消息是个成熟的概念，所以这听起来很合理。我想，我得好好考虑一下……多模型翻译对比

I'm not sure if you have already resolved the issue of flickering. I have tried many methods, including the ones you mentioned here. However, I still cannot solve the problem of my image flickering. If you have successfully resolved the flickering issue, could you summarize the solution? Thank you very much for sharing!

[miredirex]

I managed to solve the flickering issue in my iced app, thanks to @edwloef for pointing in the right direction

If frames/images arrive faster than they can be uploaded, you get flicker. Use image::allocate() to ensure frames are fully GPU-ready before displaying them, and store an Allocation instead of a Handle in the state. In the view use allocation.handle() to finally present the frame. If you're implementing a video player or an animation of some sort, abort pending allocations when new frames arrive to avoid queuing stale work

I'd suggest checking out the gallery example

Oh and also, the allocation API isn't available if you're using image-without-codecs which seems like an oversight

[ArtHome12]

@edwloef I've encountered a problem where as the bitmap size increases, starts displaying a blank screen. Is this because I'm creating Handle::from_rgba() more often than the previous implicit background image::allocate task completes?