Provide a more complete example for jaq_core

[the80srobot]

The example code in jaq_core is incomplete and not practically usable:

• It doesn't load the standard library (and basically tells you to figure it out yourself)
• It doesn't explain concepts like Arena, or what the loader does, etc
• It doesn't say where errors can arise at runtime and how to catch them

Just a tiny bit more explanation would go a long way.

[01mf02]

I agree with you. Would you like to contribute by expanding the example code / documentation in a way that you see fit?

[the80srobot]

Hi! I actually gave up on that approach and what I did instead is basically clone the entire jaq crate, removing the main function and calling real_main directly as needed.

It's a little hard to tell with things changing now around 3.0.0-alpha, but it looks like you might need a larger refactor to make the language embeddable the way e.g. lua is.

But it looks to me like that refactor might actually just be:

• Make a crate called something like jaq-embed
• Do what I did, remove main and turn real_main into the entry point.
• Rename the Cli struct into something like Options and make it slightly nicer to use programmatically
• Then just change jaq to be a layer on top that

Does that seem completely off and like I missed something about the architecture? Or does it sound reasonable?

[the80srobot]

Poking around the internals a little more, I think two current design decisions are really limiting.

• One is the assumption that the filter will output directly to stdout. This is mostly just baked into jaq and not any of the other crates, and could probably be changed so that the API takes a impl Write.
• The second one is harder to change and that is the decision to basically not support --slurp. (It's kind of supported, but doesn't combine the inputs into one array, so it's basically the same as calling the filter multiple times.) This might make sense for the CLI, but it's complicating when you're trying to use it programmatically.

So to amend, I think the intermediate crate jaq-embed would need the following modifications:

• Take all the options normally provided on the Cli + an impl Write for output. The Cli struct is already fairly easy to use, it would need only a few minor conveniences, like maybe automatically serializing struct arguments from Rust.
• Allow real slurping of inputs, so that batched processing is possible.

I think then you'd end up with an API that can run a jq program programmatically with real ease and allow you to capture output without blocking off stdin globally.

[the80srobot]

(In case it's not obvious, I am offering to do that when I have a spare weekend.)

[01mf02]

@the80srobot, it would help me to better understand your use case for jaq's API. Do you want to make a CLI, for example? Where should your inputs come from? Where should your outputs go? Are you running user-provided (arbitrary) programs, or only fixed ones?

I have started some work in #375. My current vision is to make most building blocks of the jaq crate available in some crate (currently jaq-bla in lack of a good name). I'm not sure whether CLI parsing should go into that crate, but many other things currently in the jaq crate should.

[01mf02]

@the80srobot, I have now advanced quite a bit in the direction you described (#375), by introducing the new jaq_ext crate. A relatively minimal example is: https://github.com/01mf02/jaq/blob/b6d167e7187a893029f4fcc540f7cc4ae494a9ec/jaq-ext/examples/main.rs
What do you think about it?

[01mf02]

#375 has landed and addresses this issue.