Pick the latest ubuntu version for GPU images

[bfirsh]

There are various different Ubuntu versions for the Nvidia images (-ubuntu18.04, -ubuntu20.04, etc). Cog non-deterministically picks these, instead of the latest. It should pick the latest, or at least be consistent.

[marcindulak]

I believe you refer to the https://hub.docker.com/r/nvidia/cuda/tags images, right?

Pinning the dockerhub tag may be insufficient to ensure reproducibility, since tags are mutable: new images are being pushed and they overwrite the old image tags. This creates the situation described for example here. A solution for reproducibility may be to mirror the images into controlled registries, to make sure the old images are not removed, and pin to image hashes, not tags, or maybe use immutable registries and come up with a tagging strategy that supports the new image builds. Most projects consider this an overkill, and leave the exercise of mirroring the images and finding the image pinning strategy to the users.

[anish-sahoo]

Closing as CUDA base image selection is now deterministic, choosing the latest matching Ubuntu image.