Feature/Enhancement request: Implement web-based submission flow for all browsers

[BRUrban]

Hey, I was hoping to start contributing a bunch in the new year, however I deliberately stay away from google products, and am on Linux so using Edge or Chrome is not viable for me. It would be very nice if there was a web-based flow which allowed submission from other browsers.

Specifically I'm using Firefox but it would be good to have things built in a way where they could work in any browser (opera, brave, etc).

I'm assuming the previous Import Buddy flow which is in the project wiki (https://github.com/TheDiscDb/data/wiki/Contribution-Guidelines#import-steps) still works, and will give it a whirl sometime in the next few days, but browser-based being functional at least in the 3 "main" browsers would still be very nice and probably increase contributions to the db a lot.

[lfoust]

Unfortunately, I don't control which browsers support certain APIs. To calculate the disc hash the browser needs files system access - which Firefox doesn't support. ImportBuddy still works but probably won't receive much support going forward.

[BRUrban]

Ah, yeah to be clear I was not requesting you find a way to use the Filesystem API in Firefox, The intent was to ask that a method to contribute which allows submission via Firefox be built, which presumably would need a different approach. As a bonus any approach which worked without filesystem api would most likely work for additional lower-usage browsers.

How stable is ImportBuddy? I'm pretty bummed to hear that there aren't plans to actively support the contribution method that would actually work for me, and if as a result it's likely to be brittle and require frequent massaging as a user I probably won't bother trying to contribute for now.

[shayypy]

I've been working on disc hashing and I think this might be possible via the web File API, if you're open to giving users slightly more in-depth instructions. I built a proof of concept here: https://github.com/shayypy/ddb-hash-poc (regrettably I am not hosting it myself - only needs to run for a bit at a time anyway)

Furthermore I had another idea. Since the user has to run a console command to upload MakeMKV logs anyway, could we not do something like curling a script that lists the drives on the system and uploads the hash from there? Similar to development tools which offer an installation option via a shell script.

[lfoust]

I am not sure I understand what the demo is showing. The current functionality uses the web file API but it detects how to calculate the hash from the root of the disc. The demo you linked to seems to require the user to navigate to a specific place on the disc based on if it is blu-ray or DVD? That seems more complicated for the same functionality?

As far as for having a shell script do the hash - the way I designed the current disc log upload flow was to minimize or eliminate any "magic" functionality hidden in a script. If I am asking the user to copy paste a command and run it on their machine I want it to be very clear what they are running. That is, in my opinion, one of the drawbacks of the previous ImportBuddy approach - people are being to asked to run and trust something that isn't super clear what it is.

Also, having the hashing step be separate from disc upload allows for detecting whether or not the disc is already in the database before requiring the user to upload anything. This is not something the current implementation does, but the plan is to make it clear that the disc they want to add is already in the database to avoid duplicate work.

[shayypy]

The demo you linked to seems to require the user to navigate to a specific place on the disc based on if it is blu-ray or DVD? That seems more complicated for the same functionality?

Correct, but this works on firefox, which apparently does not implement the required APIs for the current exclusive method. It's definitely a drawback and is far from perfect for many reasons. Of course, if this were implemented, it would be a fallback.

minimize or eliminate any "magic" functionality hidden in a script. If I am asking the user to copy paste a command and run it on their machine I want it to be very clear what they are running

For sure. Usually you see "read source" links with these prompts but I understand many people won't read or understand them. I personally think it's an acceptable alternative for people who can't or won't use Chrome (probably technical users from the start) but I get your concerns.

[lfoust]

I would be willing to consider a PR that has a fallback that works in more browsers - but the audience of people who won't use Chrome or Edge AND want to contribute seems to be fairly small - so not a huge priority for me.

[mihawk90]

I just stumbled on this looking for something else, but I am wondering about the premise of this issue.

When I check for availability of the Filesystem API, this has been available in Firefox since FF50:

https://caniuse.com/mdn-api_filesystem
https://developer.mozilla.org/en-US/docs/Web/API/FileSystem

Or is this using the older File API? This is seemingly deprecated and potentially subject to removal:

https://caniuse.com/filesystem
https://www.w3.org/TR/file-system-api/

So if this is still using that, it would likely be wise to use the newer API instead which would solve the originally reported issue and also futureproof it on Chrome/Edge.

That being said, I haven't tested this, but wouldn't it also work on Chromium? Reading OP it sounds like Brave and Opera are OK (though personally I fail to see how any of them are any better than Google, but that's not the point), so Chromium should be too?

---

On a (somewhat?) related note I am wondering if it's even necessary to produce a hash of every file(size?) on disc.

The KEYDB.cfg used in libaacs only uses a SHA-1 hash of a single file as a lookup key for decryption purposes. If it's good enough for decryption, it should be good enough for a database lookup, no?

If only a hash of a single file is needed, a submission might not require Filesystem access at all (which I'm sure some users might have issues with anyway). It would be possible to have a user open only the required file and the SHA-1 could be calculated in the browser.

Of course that assumes the user selects the correct file, which... might be its own downside 🤷 And would require something equivalent for DVDs.

[BRUrban]

I would be willing to consider a PR that has a fallback that works in more browsers - but the audience of people who won't use Chrome or Edge AND want to contribute seems to be fairly small - so not a huge priority for me.

While I'm not trying to claim there's an army of people with my use case just off screen, I do think it's very important to keep in mind that feedback from your existing userbase is always going to be skewed to prioritizing the features it already has. People who try to use the tool and find it doesn't work for what they want are much more likely to just silently stop using it than to engage in discussion regarding the missing functionality. I would imagine that if you enabled a change that supported firefox, you would start seeing an increase in people who are trying to use firefox to contribute to the project.

[shayypy]

I just stumbled on this looking for something else, but I am wondering about the premise of this issue.

When I check for availability of the Filesystem API, this has been available in Firefox since FF50:

It looks like DDB actually uses the File System Access API, which currently only works on chromium-based browsers: https://caniuse.com/native-filesystem-api

And in particular, showDirectoryPicker (source)

It looks like the File and Directory Entries API, which you mentioned, might be a suitable replacement/fallback. Unfortunately I will not be able to create that PR since I don't really understand this codebase but I might be able to create a demo in the future in JS if it proves to be acceptable.