Splitting out the list of providers in a standalone JSON file

[SylvainCorlay]

Currently, the available providers are listed in a JavaScript object that is defined inline in the source code of leaflet-providers.

It would be really useful to make that information available directly in a JSON file that could be used from other projects. For example, xyzservices extracts that information from leaflet-provider.js to make the same information available to Python. Xyzservices has some adoption in the scientific Python ecosystem (Geopandas, ipyleaflet, contextily, leafmap, bokeh, and others) - meaning that this project has in fact a lot of indirect adoption...

This separate JSON file could even be published independently of the leaflet-providers JS source, which would probably make it valuable for other users.

[brunob]

Hi @SylvainCorlay nice idea, we just have to find a way to generate automagically this JSON file from the repo. Do you have any hint for that ? @jieter any thoughts on this ?

[martinfleis]

Hi @brunob, we already have that magic in xyzservices. We scrape your JS to get that JSON but the result is slightly adapted to work within our use case. But the Python code for that is here https://github.com/geopandas/xyzservices/blob/main/provider_sources/_parse_leaflet_providers.py and a resulting JSON here https://github.com/geopandas/xyzservices/blob/main/provider_sources/leaflet-providers-parsed.json.

[brunob]

@martinfleis do you think we can use your script on our repo with a github ci or anything like that ?

[martinfleis]

@brunob Yeah, we do that within this GHA https://github.com/geopandas/xyzservices/blob/main/.github/workflows/update_providers.yaml using this makefile

[brunob]

Nice, let's hope that @jieter could look into it or propose an alternative method :)

[SylvainCorlay]

We could generate this JSON as a one-off, include the JSON in your repository, and then read the JSON from your JavaScript instead of having the object hard coded in the code base.

This would make it a single source of truth.

[jieter]

Yes, it would be nice to maintain the list in a machine readable format, and generate leaflet-providers.js from it + a template. I am not sure about JSON though. In the current definitions, we use some comments: (1 2 3 4), which I think is a valuable capability not possible in classic json.

Other options would be JSONC, but that would require consumers of the file to use non-standard parsers too.

Any suggestions?

[martinfleis]

What about using YAML instead if we need comments? That feels as a bit more standard solution than JSONC.

[SylvainCorlay]

The big advantage of JSON is that you can parse it from either Python or JS without a third-party dependency.

[brunob]

+1 for JSON too

[martinfleis]

One option to keep comments in the JSON without using non-standard parsers would be to include an optional note or comment field with a string that is currently in the comment and ignore it while parsing the JSON to show it on your website or within xyzservices.

[rafaqz]

We would also like this at TileProviders.jl (which is used in Leaflet.jl but is generalised so everyone can use it)

Currently we pull json from geopandas xyzservices because its easier to parse

To minimise downstream workload I suggest that rather than YAML or a new JSON format we use the JSON format xyzservices has already defined.

[vincerubinetti]

This would be great, and looking at the code I can't imagine it would be too hard to split it out. I'd be happy to contribute a PR, but it looks like there's a lot of old unmerged PRs.

I've switched from Leaflet to Open Layers for a variety of technical reasons, but there doesn't seem to be an equivalent plugin like this one for OL.

Currently it looks like the geopandas extraction of the JSON here is more up to date, so I will graciously take that and import it into my project 😇 It'd be great to just have it right from the source though.