C#: Extract string interpolation alignment and format.

[michaelnebel]

This PR fixes an issue with missing extraction of alignment and format clauses from string interpolation expressions as described here.

Prior to this change, when we extracted a string interpolation expression $"Hello {name,align:format}. Welcome." (denoted E in the graphs below), it looked like this in the expressions relation

graph TD;
  id1["#quot;Hello #quot;"]
  id2["name"]
  id3["#quot;. Welcome.#quot;"]
  E--0-->id1
  E--1-->id2
  E--2-->id3

Loading

With this change we introduce a new expression for `{name:align:format}, which we call a string interpolation insert expression. The expression tree for extracting a string interpolation expression now looks like,

graph TD;
  id1["#quot;Hello #quot;"]
  id2["{name,align:format}"]
  id3["#quot;. Welcome.#quot;"]
  id4["name"]
  id5["align"]
  id6["format"]
  E--0-->id1
  E--1-->id2
  E--2-->id3
  id2--0-->id4
  id2--1-->id5
  id2--2-->id6

Loading

This change in the expression tree of string interpolation expressions makes it impossible/difficult to make a proper upgrade/downgrade script, as we introduce a new kind of expression and alter the parent/child relationship. That is, to make a proper upgrade script, we would need to introduce new expressions and alter- and add new entries in the expressions and expr_parent relations. We might be able to do it a bit better in the downgrade script, but I have chosen just to change the kind of the expressions to unknown. Conversely the downgrade script needs to remove some elements from expressions and expr_parent and also do so minor edge contraction on expr_parent.
Effectively, this change means that we break dataflow for string interpolation expressions, when the upgrade and downgrade scripts are used.
The up- and downgrade scripts have been tested manually on the database that can produced from the testcase. I tested that

• The parent/child relationship for inserts are adjusted correctly and new expressions are added when needed.
• The difference in the size of the expressions and expr_parent are as expected, when applying the upgrade and downgrade scripts (this is a cardinality test to give an indication that we don't add or drop to many elements from the tables).

Other relevant implementation details

• The new expression kind should just used the standard post order control flow.
• Taint flow for string interpolation expressions are now split in two steps, as we need to propagate taint from name to {name,align:format} in the example.

DCA looks un-eventfull.

[hvitved]

LGTM, great work. Some small questions.

[hvitved]

Remove the trailing comment?

[hvitved]

Remove the trailing comment?

[hvitved]

Should this select the insert part of e instead of e?

[michaelnebel]

I considered doing that as well, but it requires that we also make changes to the expr_parent relation as well (this is what I am vaguely hinting in the PR description).
Basically, I am worried that I break some DB invariant by changing the parent/child relationship as a part of the script.
Should we attempt to make the better downgrade script? It requires
(1) Removing the insert expressions from the expressions relation
(2) Adjust the expr_parent relation
(3) Consider if we need to do something with the new dangling align and format expressions.

[alexet]

to make a proper upgrade script, we would need to introduce new expressions and alter- and add new entries in the expressions and expr_parent relations

This is possible using the NewEntity builtin module (only available in upgrade and downgrades) and using a single file upgrade query. Obviously the alight and format can't be created out of nothing though.

[michaelnebel]

to make a proper upgrade script, we would need to introduce new expressions and alter- and add new entries in the expressions and expr_parent relations

This is possible using the NewEntity builtin module (only available in upgrade and downgrades) and using a single file upgrade query. Obviously the alight and format can't be created out of nothing though.

Uh, I was not aware of that! Thank you!

[michaelnebel]

@hvitved : Proper upgrade and downgrade scripts have been added, since your last review and you review comments have been addressed (the last three commits).