materialize JSON-LD versions of the manifests

[pchampin]

note that these versions are generated internally to generate the HTML versions. This change simply writes the results in a file.
Some people might find this more convenient to process than the TTL files?

I don't feel strongly about this,
I ended up doing this while getting myself familiarized with the automation process.

[Tpt]

Nice addition for implementations that might want to run some tests without a proper Turtle parser

[afs]

Rather than JSON-LD in the tree (FWIW they used to be produced), I think it is better to produce the jsonld files as the GH-pages are created. Then there is one version of the truth.

[pchampin]

Rather than JSON-LD in the tree (FWIW they used to be produced), I think it is better to produce the jsonld files as the GH-pages are created. Then there is one version of the truth.

I sympathize with that, and to mitigate the potential issues, I added a "comment" at the top of each JSON-LD file, explaining that they are generated and should not be edited.

Arguably, the whole generation process could be restricted to the page generation, rather than materialized in the repo. Or do you consider that the HTML pages are different from the JSON-LD files?

[afs]

I sympathize with that, and to mitigate the potential issues, I added a "comment" at the top of each JSON-LD file, explaining that they are generated and should not be edited.

I saw that.

Arguably, the whole generation process could be restricted to the page generation, rather than materialized in the repo. Or do you consider that the HTML pages are different from the JSON-LD files?

The design I see as best for the long-term is that the repo is the primary copy of the tests.
It should survive having to move again.

There is a website that is a presentation of that content for certain audiences. It is produced from the repo files with HTML, copies of the all the files and JSON-LD manifests.

[pchampin]

There is a website that is a presentation of that content for certain audiences. It is produced from the repo files with HTML, copies of the all the files and JSON-LD manifests.

as a nice-to-have, we should probably also release zips containing all the files, including the generated ones... (again, some people might be interested in processing the JSON-LD manifests rather than the TTL ones, and still do that from a local copy)

Do you think this change (moving from "in repo generation" to "in pages generation") should be included in this PR before merging?

[Tpt]

as a nice-to-have, we should probably also release zips containing all the files, including the generated ones...

Seems fairly easily alongside GitHub pages publishing. I will give it a try today

[afs]

Do you think this change (moving from "in repo generation" to "in pages generation") should be included in this PR before merging?

I won't block but this PR is all about adding the JSON-LD manifests to the repo. I do not support that.

[Tpt]

MR that stops to push HTML pages but publishes them to GitHub pages: #255

[afs]

we should probably also release zips containing all the files,

Github provides download as zip.

https://github.com/w3c/rdf-tests/archive/refs/heads/main.zip

[pchampin]

I won't block but this PR is all about adding the JSON-LD manifests to the repo. I do not support that.

That's fair. If #255 is accepted, once merged, I will make a new PR with only the changes on Rakefile, so that the JSON-LD versions of the manifests will only be present on github-pages and the released ZIP.

Github provides download as zip.

Yes, but if we restrict the generation of HTML and JSON-LD manifests to github pages, these ZIPs will only contain the TTL manifests. Some people might prefer that, but some people may want the full set of files, including the generated ones.

[afs]

Some people might prefer that

Please substantiate "some". We simply can not accommodate every possibility we can think of. Why can't we assume that users can convert using local or online tools if necessary for them? This repo contains Turtle and N-Triples tests. They are likely to have a TTL parser.

The tradeoff is having one version of the truth.

And it does get out of step. GH actions fail, then the newest tests are in the TTL and not the JSON-LD yet they are the focus at the time.

[afs]

By "release zip", do you mean the built state at a point in time with a version number?

[Tpt]

Please substantiate "some".

I have one use case: when writing the firsts NTriples and Turtle parser in Rust for Oxigraph, I would have liked a JSON version of the manifest to parse it with an existing JSON parser and get the testsuite for NTriples to validate my NTriples parser without having built my Turtle parser yet.

And it does get out of step. GH actions fail, then the newest tests are in the TTL and not the JSON-LD yet they are the focus at the time.

I think the idea of not committing the JSON-LD files but building them on the fly when publishing the website is a good workaround: if the action fails the JSON-LD files are just not generated and the website not published.

[Tpt]

I think that we can take the same approach as the HTML files: do not commit them in the main branch but in the gh-pages branch. For that I think we just need to tweak the git add calls in .github/workflows/generate.yml to also commit the jsonld files.

@pchampin I can take over the MR to get it done if you want

[pchampin]

@pchampin I can take over the MR to get it done if you want

that would be great, thanks :)

[Tpt]

I have rebased the MR. It now generates the JSON-LD files in the same gh-pages branch as the HTML files. I also apply to the JSON-LD files the preprocessing done for HTML templating. This way we "dogfood" them when generating the HTML pages.

@afs @pchampin @rubensworks new review welcome!