Pipe data into and out of CLI

[josephjclark]

I should be able to do something like this:

echo "{ \"x\":1}"  | openfn -S -a common tmp/echo.js -O | jq .data

But -S isn't really set up properly, and I'm not really sure how to pipe output properly. Maybe I need to suppress all logs when used in this mode? I want to just pipe the output JSON

Today you can:

• Pass -O --log none to return to stdout and supress logs. That works
• Pass state into stdin openfn -S=$SOME_JSON echo.js

[josephjclark]

Consider logging as standard to console.error, so that when piping only the final output goes to the next process

The -O --log=none thing actually doens't seem to work inless I replace logger.always with console.log.Which makes sense but I'm sure this actually worked a moment ago!

[josephjclark]

Because I'm scared of breaking changes I might add a special flag which means "pipe mode": ie log to stderr and output json. That would give a cheap workaround while I consider whether we want to always log to stderr