Question: whitelisting / retrieve applicable mirror list in machine-readable format?

[jmozd]

Hi,

we were asked to provide URL whitelists for Internet access, and regarding openSUSE downloads would like to automate this by querying MirrorCache.

I'm aware of using i.e. "https://download.opensuse.org/distribution/leap/15.6/iso/openSUSE-Leap-15.6-DVD-x86_64-Build710.3-Media.iso?mirrorlist" to receive a currentlist of potential mirrors in human-readable format. Is there some query parameter I may add so that i.e. a JSON is returned?

And with regards to the list of applicable mirrors returned - comparing to "mirrors.opensuse.org" I can tell that above URL only returns a subset of mirrors. And as I see at least a few connects to mirrors not contained the the result from above URL (but contained in mirrors.opensuse.org) I must assume that the "?mirrorlist" query will return changing entries across multiple queries. True?

How are others handlng whitelisting requirements?

[andrii-suse]

Yes, you can add json parameter to that url

https://download.opensuse.org/distribution/leap/15.6/iso/openSUSE-Leap-15.6-DVD-x86_64-Build710.3-Media.iso?mirrorlist&json

It returns three arrays: country, continent, rest

[andrii-suse]

The content on mirrors is not controlled by any centralized entity, so download.opensuse.org runs various scanning jobs to react to actual state of files on each mirror. There might be some delay in that detection, which might be one of the reasons of the difference you see.

Another thing is healthcheck jobs, which might exclude unstable/unavailable mirrors from the mirrorlist output.

If you see any difference which doesn't fall into reasons mentioned above - please provide an example, so I would try to troubleshoot investigate the reasons.

[jmozd]

If you see any difference which doesn't fall into reasons mentioned above - please provide an example, so I would try to troubleshoot investigate the reasons.

I retrieved both https://mirrors.opensuse.org/ and https://download.opensuse.org/distribution/leap/15.6/iso/openSUSE-Leap-15.6-DVD-x86_64-Build710.3-Media.iso?mirrorlist&json

In the json result for the ISO, there's only mirror.rackspace.com, while https://mirrors.opensuse.org reports mirror.rackspace.co.uk, mirror.rackspace.hk and mirror.rackspace.com as mirrors for 15.6 ISO, all there color-coded as "up-to-date".

In the json result for the ISO, opensuse.teambelgium.net is named, but mirrors.opensuse.org does not mention it at all.

I see repo.ugm.ac.id and repo.iut.ac.ir in mirror.opensuse.org (for 15.6 ISO, green), but not in the json output. There might be more, I didn't do a full backward check.

[andrii-suse]

opensuse.teambelgium.net was a special case, (because it is responding only over https - it was mistakenly reported as problematic in some places) it should be fixed now.

Regarding the others - it looks the mirrorlist is too strict, not showing the IPv4 - only mirrors when request is done over IPv6

I see that such logic can be be beneficial in some cases, but still feel free to report a bug for that. Or request an additional parameter that ignores that IPv strictness.

Meanwhile to get the list for both IPv4 and IPv6 mirrors - one will need to do additional request over IPv4, e.g.

# curl -s4 'https://download.opensuse.org/distribution/leap/15.6/iso/openSUSE-Leap-15.6-DVD-x86_64-Build710.3-Media.iso?mirrorlist&json' | grep -o rackspace.hk
rackspace.hk

or just use IPV4 parameter:

# curl -s 'https://download.opensuse.org/distribution/leap/15.6/iso/openSUSE-Leap-15.6-DVD-x86_64-Build710.3-Media.iso?mirrorlist&json&IPV4' | grep -o rackspace.hk
rackspace.hk