Tracking issue for unstable `--json-rendered` flag

[alexcrichton]

This issue is intended to be a tracking issue for the --json-rendered flag implemented in #59128. The --json-rendered flag can either take plain or termcolor, and affects whether ANSI color codes are embedded into the rendered field of JSON diagnostics.

Known issues with this:

• flag not supported in rustdoc yet

Future features:

• Html output (discussion: Emit ansi color codes in the rendered field of json diagnostics #59128 (comment))
• Short output (equal to --error-format=short)

cc @oli-obk, original author of the feature

[alexcrichton]

I'd like to propose that this feature is stabilized in an effort to stabilize pipelined compilation. As a brief summary of how this flag works today, when passed to the compiler the --json-rendered flag will affect the "rendered" field of JSON diagnostics printed by the compiler.

I would propose the following stabilization:

• Add a new flag, --json-rendered
• Hide this flag from rustc -h unless the --verbose flag is passed
• Accept the following values for --json-rendered=value:
  • plain - whatever rustc would print on the console but without colors
  • ansi-colors - whatever rustc would print on the console with ANSI color codes embedded in the string
  • short - whatever rustc would print on the console for --error-format=short, but without colors
  • short-ansi-colors - combination of the two options above

This should be sufficient for all of Cargo's usage, and we'd mirror the option into rustdoc as well as rustc for processing there.

[alexcrichton]

Oh and cc @rust-lang/compiler, could I persuade one of y'all to kick of FCP for stabilization? I wrote a summary above for what I'm proposing to be stabilized here.

[eddyb]

I feel like we're accumulating a bunch of flags that are related but not explicitly grouped (like -C flags are), maybe there is a better way to set this up? Other than that I'm on board!

[oli-obk]

Instead of having a a non-color flag and a color flag for each variant we can also reuse the --color flag that modifies how non-json output is colored.

[jsgf]

Would it be possible to have an option to include all rendering styles in the case where it is being cached and we don't know how its going to be used or consumed?

Is color vs. non-color just a matter of stripping out ctrl characters?

[eddyb]

@jsgf With the ANSI output, you'd need to have a small parser to find where the escape sequence stops (i.e. it always starts with a \x1b but can be followed by a varying number of characters).
While possible, it might be too much work, so it could be a good idea to have a different field, rendered_ansi_colors which contains the colored version.
This would also be backwards-compatible, if that's a concern.

[ehuss]

@eddyb Cargo already implements ansi parsing and stripping when necessary. It uses the strip_ansi_escapes package for stripping and fwdansi for converting to old-Windows color controls.

[ehuss]

I'm a little concerned about stabilizing this without having a strategy for supporting the caching use case. It needs a single JSON format that can be converted to all other output formats. Currently it works for everything except short messages. I see a few options here:

1. Support ability to include multiple rendered formats in JSON, which could influence the format of the --json-rendered flag.
2. Use an external library for rendering messages, and share that between rustc and cargo. This would probably require a new JSON format altogether. Once this is in place, cargo could stop using --json-rendered.

ping @rust-lang/wg-diagnostics. I'm curious what the feasibility of 2 is. Is it on the table as an option? Is it something that can happen within a year?

I also think it's a little strange to have separate color controls from the --color flag.

[jsgf]

@ehuss Yes, that's exactly my concern - I'd like to cache the json output and be able to generate any of the other forms on demand, without having separate implementation of diagnostics rendering.

[oli-obk]

@rust-lang/wg-diagnostics is working on integrating the annotate-snippet-rs crate into rustc and making it the go-to rendering engine (see #59346). Implementing a json -> rendered scheme is not on the table at the momemnt, but I see nothing that would speak against it. All relevant information should be available.

We could even extend --json-rendered to also have a not/none variant that just produces null for the rendered field (or an empty string) in order to reduce the amount of data in json that you don't need.

So it definitely is doable, but we have our hands full with just using annotate-snippet-rs in the compiler at all.

[alexcrichton]

@jsgf and @ehuss can y'all explain more why you think that a diagnostic JSON message should contain all possible renderings? I understand the idea of wanting it cached, but why would you want it cached and dynamically switched between builds? This seems like a setting that is rarely flipped and wouldn't really benefit from an A+ user-experience of caching as much as possible.

I would personally be fine interpreting --color always as simply "put ansi codes into the json rendered field". This is largely a means to an end of pipelined compilation, and I wish that nothing had to be stabilized anyway.

To that end could I propose that we simply stabilize the intent of interpreting --color always as "put ansi codes into the json rendered field" and table the question of caching short/long at the same time?