extra images are sometimes not deleted after abort

[teracow]

Might be time to look at trapping an abort signal? Maybe use SIGABRT 6 ?

Then modify the abort routine to send 6 instead of 9 to running jobs.

[teracow]

OK, now using SIGABRT, but finding that I need to background each image download command, then grab its PID, then kill that PID if it turns out we're going to abort that download. Then delete any partially downloaded image.

This is becoming quite a "thing". Ugh!

[teracow]

Also need to save output from backgrounded downloader regarding exit status and messages.

[teracow]

Ugh... I can't reliably catch and kill each background process and ensure it hasn't already downloaded an image, but not yet marked it as complete. There's a narrow window of time where this can occur.

I think I'll work on downloading images into temp files, then renaming those we want to keep. The temps can be deleted afterwards. The gallery builder should be coded to only work with non-temp image files. This should be a more consistent method.

This is my first project dealing with asynchronous processes. It's been a learning curve. 😁

[MBtech]

What the scenario here? That the user aborts the run?

[teracow]

What the scenario here? That the user aborts the run?

Yes, but aborting also happens during a regular run. That's the one I'm testing. When we have enough images, all the backgrounded downloaders are aborted, and we then need to cleanup any partially (or sometimes fully) downloaded image it may have created.

I'm hoping to have a fix published for this in the next few hours.

[teracow]

today's progress: getting there. 😄

I've written from scratch a new and quite simple version of the downloading functions used in googliser. For testing, it can run upto 500 instances of itself, each downloading the same remote file and saving it locally under different names. In operation, it launches as many forks as it can in 10 seconds (which turns out to be all 500 of them), and each one starts a backgrounded instance of curl (so it can be killed by the parent function when a SIGABRT is received).

It allows the downloaders to run uninterrupted for 10 seconds, then issues a SIGABRT to simulate the signal issued when googliser has received enough valid images. This instantly kills each copy of curl, and the parent function cleans-up any partially downloaded files left by curl. This is exactly how I want googliser to behave. 😆

So far, it's working flawlessly, but is still being developed and is not yet ready to transfer into googliser. I'm hoping I'll get a chance to work on this over the Christmas holidays, along with fixing the brakes on the car, repairing the garage roof, etc...