version migrator changes tarball extension

[h-vetinari]

I've seen this in a few LLVM feedstocks:

 source:
-  - url: https://github.com/llvm/llvm-project/archive/refs/tags/llvmorg-{{ version.replace(".rc", "-rc") }}.tar.gz
-    sha256: 933765a1c2cd518d95a9033a92d88d7109a79aefa4609247c31f28b8bc8dd96e
+  - url: https://github.com/llvm/llvm-project/archive/refs/tags/llvmorg-{{ version.replace(".rc", "-rc") }}.zip
+    sha256: f2c42f5a28cc74b7eff64e560ad4ecabf42be9dd948f2b8c7fef3173a76363dc
     patches:

That is not desirable IMO. CC @beckermr

[beckermr]

Ok. The bot does this because sometimes people do change extensions and then we miss updates.

Suggestions for how to proceed? Maybe a config option?

[h-vetinari]

Suggestions for how to proceed?

Prefer the existing extension if it exists?

[beckermr]

Yep that is the current default. The issue is that sometimes the hashing of the original extension fails due to network outages. Then the network comes back and the bot finds the new extension since now it is checking that one.

This happens a lot more often than I naively expected. It is actually one of the main reasons the bot test suite is flaky.

[h-vetinari]

IMO it'd make sense to spend the runtime to do a couple retries internally with an exponential (but still short) backoff, rather than chase after all these flakes and random other associated changes. Something like

Try	Increment	Total Time
1st	-	0 sec.
2nd	2 sec.	2 sec.
3rd	4 sec.	6 sec.
4th	8 sec.	14 sec.
5th	16 sec.	30 sec.

That's a worst case of 30 sec. lost, which however would only happen only if the source is gone completely, and that's very rare. If that's still too much, then going just until the 3rd try would already catch a guesstimated 90+% of failures.

I'm only talking about existing sources (and their file extensions); trying new extensions doesn't need a backoff.

[beckermr]

We already retry as far as I know.