[objc2_av_foundation] Trying to read QR codes on macOS, can't get it to work

[miried]

So, not wanting to off-topic your issue tracker, but I'm trying to get a small sample working, which is relying on objc2 crates. I basically open a capture session with objc2_av_foundation and the Macbook camera as input, and read any QR codes as output. It is kind of following the sample code from Apple, which is for iOS, but I tried to distill it to only use the bare functionality to set up what I need, and as far as I can see, all the functionality I'm using should be available on macOS 13+.

Is the MetadataOutput supposed to work on macOS, or only iOS? Am I doing something wrong? See issue here: miried/avf-qr#1, and sample code here: https://github.com/miried/avf-qr/blob/main/src/main.rs.

Thanks for any help!

[madsmtm]

So, not wanting to off-topic your issue tracker

Not at all, feel free to ask questions!

I think the panic is from calling setMetadataObjectTypes with AVMetadataObjectTypeQRCode that it says with availableMetadataObjectTypes that it doesn't support. If you do the following instead:

av_capture_metadata_output.setMetadataObjectTypes(Some(&available_types));

You should see the camera turn on.

As to exactly why that is, dunno, but maybe that'll help you along?

If you run into further issues, I'd suggest you try to bundle your app with a PList that has NSCameraUsageDescription set, and/or that you try to run your code in a run loop.

[miried]

Thanks for your suggestions. I added a run loop and bundle information with NSCameraUsageDescription as per your suggestions. (see https://github.com/miried/avf-qr/commits/main/)

The run loop doesn't seem to change anything. The camera lights up. But it did already with my original code.

Note that availableMetadataObjectTypes is empty for me. That made me wonder if macos camera is considered "dumb" and no meta data output is available?

When I run it as an app bundle (instead of CLI command), it asks me for permission, and when I click, the app runs for the defined time, but without camera light turning on. When I remove setMetadataObjectTypes, then it runs, and camera light turns on, but nothing else happens, and the app stops running after the waiting time.\

I don't know how to debug-print when I run it as an app, unfortunately.

Any more ideas welcome 😄

[miried]

No luck AVCaptureMetadataOutput so far, but I tried to connect the video output with the QR code detection in CoreImage & barcode detection in Vision frameworks. At least, these two both work on macOS Sonoma.

The AVCaptureMetadataOutput seems to be available in general on recent versions of macOS (seemingly introduced with the "Continuity Camera" feature according to this WWDC2022 presentation. So I'm wondering if it is only useful in connection with this feature (using the phone camera on the macbook) or on iOS.

I'm not expert enough with this stuff and since I get detection with CoreImage and Vision, it works for my purpose. If anyone makes it work though, I'd be curious to know the solution.