Updated context gets passed *all the way* downstream.#142
Conversation
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
There was a problem hiding this comment.
The only case where this is a bit tricky is where you have two parallel nodes.
Suppose you have Node A -> (Node B, Node C) -> Node D
Because we run Node B and Node C sequentially, Node C will see the updated context from Node B.
But I think this isn't an issue, because later we enforce that all parents of Node D must produce the same updated context, so I think it's safe?
There was a problem hiding this comment.
The code is actually smart enough right now to handle the parallel case as well! Because when we update the context, we only pass the outputs from that node's predecessors. And node B is not a predecessor for node C.
I added some more tests to confirm that things work as expected.
Example: node A -> node B -> node C.
Assume node A updates the context.
Before: node B would see that updated context, but node C would only see the original context.
Now: the updated context is floated all the way down stream (or until it's updated by another node).
Before, this was a problem in our security DAG; the refusal detector would see the decoded sut response, but then the safety arbiter would only see the original, encoded response.
Let me know if my fix has consequences that I am not thinking of.