From af1aaa898a373a434184411a1608624c948f019d Mon Sep 17 00:00:00 2001 From: Vikhyath Mondreti Date: Mon, 10 Aug 2026 08:18:25 -0700 Subject: [PATCH 1/7] feat(library): What Is an Agentic Workflow? (And How It's Different From Automation) (#6489) * feat(library): What Is an Agentic Workflow? (And How It's Different From Automation) * content(library): restore original Zapier/Make wording in agentic-workflow post --------- Co-authored-by: Sim Pi Agent --- .../what-is-an-agentic-workflow/index.mdx | 98 ++++++++++++++++++ .../what-is-an-agentic-workflow/cover.jpg | Bin 0 -> 27669 bytes 2 files changed, 98 insertions(+) create mode 100644 apps/sim/content/library/what-is-an-agentic-workflow/index.mdx create mode 100644 apps/sim/public/library/what-is-an-agentic-workflow/cover.jpg diff --git a/apps/sim/content/library/what-is-an-agentic-workflow/index.mdx b/apps/sim/content/library/what-is-an-agentic-workflow/index.mdx new file mode 100644 index 00000000000..f969fe1f70a --- /dev/null +++ b/apps/sim/content/library/what-is-an-agentic-workflow/index.mdx @@ -0,0 +1,98 @@ +--- +slug: what-is-an-agentic-workflow +title: 'What Is an Agentic Workflow? (And How It''s Different From Automation)' +description: 'Learn what an agentic workflow is, how it differs from traditional automation, and when to combine AI agent reasoning with deterministic workflow controls.' +date: 2026-08-10 +updated: 2026-08-10 +authors: + - andrew +readingTime: 7 +tags: [AI Agents, Workflow Automation, Agentic Workflows, Sim] +ogImage: /library/what-is-an-agentic-workflow/cover.jpg +canonical: https://www.sim.ai/library/what-is-an-agentic-workflow +draft: false +faq: + - q: "Is agentic AI the same as automation?" + a: "No. Agentic AI chooses actions at runtime, while traditional automation follows human-authored rules. Sim supports both modes in one workflow graph, so you can use reasoning for ambiguous work and fixed logic for predictable steps." + - q: "Can Zapier or Make build agentic workflows?" + a: "Zapier and Make can include AI or agent steps within their trigger-action workflows. Sim instead treats Agent blocks as first-class parts of the execution graph alongside deterministic blocks. Your choice depends on whether reasoning serves as one isolated step or directs several parts of the workflow." + - q: "What is the difference between an AI agent and an agentic workflow?" + a: "An AI agent reasons and uses tools, while an agentic workflow coordinates that reasoning with triggers, data, control flow, and other steps. The workflow gives an agent defined inputs, boundaries, and operational context." + - q: "Do agentic workflows replace deterministic automation?" + a: "No. Agentic workflows still need deterministic automation for validation, routing, approvals, and compliance rules. Sim lets builders combine those fixed steps with model reasoning in one graph, reserving autonomy for decisions that require judgment while keeping sensitive actions predictable." +--- + +## TL;DR + +- An agentic workflow is a system where an AI model evaluates context and chooses its next action or tool call at runtime. +- Traditional automation follows triggers, branches, and actions that a human defines in advance. The workflow cannot choose an unplanned next step. +- Agentic reasoning and deterministic automation serve complementary roles. Production systems often use reasoning for judgment and fixed logic for validation, routing, and approvals. +- Sim provides one example of this hybrid architecture. Its workflow graph combines Agent blocks with functions, conditions, routers, loops, parallel branches, and human approval. + +## What an agentic workflow actually is + +An agentic workflow is a system in which a model reasons over available context and chooses its next action or tool call while the workflow runs. Traditional automation follows triggers, branches, and actions that a person defined before execution. This runtime use of reasoning and tools is also central to [what an AI agent is](https://www.sim.ai/library/what-is-an-ai-agent-definition-how-it-works-and-examples). + +A support-triage run provides a concrete example. The model reads an email that reports both a billing discrepancy and possible account compromise, checks the customer’s account history, and then chooses between opening a billing ticket and escalating a security case. The email and retrieved account data determine the choice during execution. No human-authored branch covers that exact combination of claims and account history. + +Runtime discretion still operates within boundaries. You define the workflow’s goal, available tools, instructions, and permissions in advance. The model decides which permitted action best fits the current context instead of matching the input against a complete set of prewritten conditions. + +## What traditional automation is, and where Zapier and Make sit today + +Traditional trigger-action automation executes a path that a human defines before the run begins. You choose an event that starts the workflow, then specify which actions follow under each condition. Every trigger and branch is authored in advance. At runtime, the software evaluates those rules, but it cannot choose a new type of action outside them. This distinction resembles the boundary between agents and rule-driven systems explored in [AI agents vs. RPA](https://www.sim.ai/library/ai-agents-vs-rpa). + +Zapier and Make grew around this architecture. [Zapier’s conventional workflow builder connects triggers and actions](https://help.zapier.com/hc/en-us/articles/8496183585293-Create-Zaps), while [Make scenarios arrange modules and routes in a visual editor](https://help.make.com/scenario-editor), so builders define the available paths and the conditions that select them. Both platforms now offer agent features through [Zapier Agents](https://zapier.com/agents) and [Make AI Agents](https://www.make.com/en/ai-agents). In a standard automation, however, an AI step usually performs a bounded task inside a predetermined path, such as classifying an email or drafting a reply. Human-authored logic still determines when the model runs and where its output goes. + +Trigger-action automation cannot create a path for input its author did not anticipate. A malformed payload may cause the run to fail. An unmatched condition may follow a fallback route or end the run, depending on the authored rules. The software does not inspect the situation and select an unplanned tool. Reasoning remains a capability within a step rather than the mechanism that directs execution. + +## Agentic workflows vs. automation, side by side + +The main tradeoff is runtime discretion versus human control. Zapier- and Make-style automation follows authored paths, while an agentic workflow lets a model choose among available actions during execution. + +| Comparison | Traditional automation | Agentic workflow | +| --- | --- | --- | +| Decision-maker | A human defines each trigger, condition, branch, and action before execution. | A model evaluates the current context and selects the next action or tool during execution. | +| Unexpected input | The workflow follows a fallback path, skips a step, or fails when no authored rule covers the input. | The model can interpret unfamiliar input and choose an available action, although its choice may be wrong. | +| Flexibility and predictability | Fixed paths produce consistent behavior for known inputs but require edits when requirements change. | Runtime reasoning handles wider input variation but can produce different decisions for similar inputs. | +| Typical failure mode | Schema drift, missing fields, or unhandled conditions can break the workflow or send execution down the wrong branch. | The model can loop, invent an unsupported tool call, pass invalid arguments, or act on flawed reasoning. | +| Auditability and debugging | Each action maps directly to an authored node and condition, which makes execution paths easier to reproduce. | Debugging requires inspection of prompts, context, model outputs, and tool calls. Model variability can make exact reproduction harder. | + +Neither column represents a universal improvement. Each mode accepts a different set of constraints and failure risks. For a deeper look at tracing model decisions and tool calls, see [AI agent observability](https://www.sim.ai/library/ai-agent-observability). + +## Why agentic and deterministic are complementary, not competing + +Agentic and deterministic modes solve different kinds of steps, so forcing one mode across an entire workflow usually creates avoidable failure. A fully deterministic workflow cannot handle cases its author did not anticipate. A fully agentic workflow gives a model discretion over steps that may require exact, repeatable behavior. + +Deterministic logic belongs wherever the same input should produce the same action. A validator can reject records that lack required fields, while routing rules can send approved records to fixed destinations. Human approval gates can pause refunds or account changes until an authorized person reviews them. These controls give auditors a clear record of which rule produced each action. + +Agentic reasoning fits steps that require judgment under ambiguity. A model can interpret an unstructured request and weigh incomplete evidence. The model can then choose between searching a knowledge base, querying a CRM, or asking the user for more information. Because model reasoning can produce an incorrect tool call or enter a loop, the surrounding workflow should enforce tool permissions, retry limits, and output checks. + +A support workflow shows how the modes can divide responsibility. Deterministic validation can verify the ticket fields before an agent interprets the customer’s message. The agent can recommend a response and select a relevant tool, while fixed rules route high-value refunds to human approval. Each mode handles the failure it can control most effectively. Practical [support ticket triage patterns](https://www.sim.ai/library/best-ai-agents-support-ticket-triage) apply the same division of responsibility. + +Architecture determines how closely those modes can cooperate. Automation-first platforms such as Zapier and Make generally place AI steps inside a trigger-action graph whose surrounding paths remain human-authored, as reflected in the official guides to [adding AI steps in Zapier](https://help.zapier.com/hc/en-us/articles/16276574838925-Use-AI-by-Zapier-in-your-Zap) and [building scenarios in Make](https://help.make.com/scenario-editor). The model can reason within its assigned step, but predefined edges usually control where execution goes afterward. Builders must add branches around the AI output if they want the wider workflow to react differently. + +A reasoning-native graph treats the model as a node that can choose tools and return structured decisions to deterministic controls. Fixed blocks can validate the model’s output, restrict available actions, and require approval before execution continues. You can then give the model discretion only where the task needs it, while preserving predictable behavior around sensitive operations. + +## How Sim structures agentic and deterministic blocks in one graph + +[Sim](https://sim.ai) provides one example of the hybrid pattern. Its visual workflow graph places Agent blocks alongside deterministic blocks, so model reasoning participates directly in execution rather than sitting inside a fixed automation step. + +Agent blocks reason over available context and choose tools during a run. Builders can constrain that discretion by selecting the tools an Agent block may call and defining the structured output that later blocks receive. + +Deterministic blocks control the parts that need fixed behavior. Functions validate or transform data, while conditions and routers direct execution through human-authored paths. Loops repeat known operations, and parallel branches run independent work concurrently. Human approval can pause the workflow before a consequential action proceeds. + +A support workflow shows how these blocks can work together. An Agent block can interpret an unstructured request and propose a category. A function can validate the required fields, and a router can send refund requests for human approval while passing routine account updates to an integration action. + +Sim makes the boundary between discretion and control visible in the graph. The Agent block handles ambiguity within its permitted tools, while surrounding blocks enforce validation, routing, and approval rules. You can inspect each block’s inputs and outputs after a run, which helps isolate whether a failure came from model reasoning or deterministic logic. + +## Deciding whether a given step should be deterministic or agentic + +Evaluate each workflow step separately. A single workflow can use deterministic rules for one decision and agentic reasoning for the next. + +- Would the same input always require the same action? Use a deterministic step when you can express the decision as a stable rule. +- Must an auditor reconstruct why the step ran? Use deterministic logic when you need an explicit decision path, fixed approval rules, or repeatable results. +- Does the step interpret unstructured input? Use an agent when the answer depends on context, judgment, or meaning that fixed conditions cannot cover reliably. +- Must the step choose among tools based on the situation? An agent can inspect the request and select an appropriate tool at runtime. +- Could a wrong decision create a legal or financial consequence? Keep the final action deterministic or require human approval, even if an agent prepares the recommendation. + +You can also split one task across both modes. An agent might interpret a customer request and propose a route, while deterministic rules validate required fields and approve the destination. Good workflow design puts the boundary where judgment ends and enforceable rules begin. diff --git a/apps/sim/public/library/what-is-an-agentic-workflow/cover.jpg b/apps/sim/public/library/what-is-an-agentic-workflow/cover.jpg new file mode 100644 index 0000000000000000000000000000000000000000..8d069b7ba4d3dd571c4c4d00173197d04eb5e304 GIT binary patch literal 27669 zcmeFYWptZOvMwq!Gh@sQF>}n!%*@Qpj+vR68Dxf-*)cQ6j7jX6F<1HKo0)UhzIUxX z=bWGWm9!+a-mb2$s;+vfy7hbQ_a;CT0R4{(1_l}q9`ry$L_k19!bC;^Jup9DU}J!u z9|%YZ@IWsXDl#%E7H(!nW^OS70Wl3_4JW7cKSaR(kBJNd&|tw!AR8gT$N=DIU=V0v zzlQ;Y05AYJ1Q-|?;9nOs3?vjR1o)p{G5^>0f86kU1Aqtt>Vpb_3TkNW|5vyF_yd%n zDM8_X+)La@e5M>~rnYtt0QqspriND8*y3h;U)YiZso3$;)25d$0X{S0sU7?~mA)>q zdmeVse>TLT#7Pq?0?I@RLVjjvKR{ihzV@?rYHwlNJ;!vh^2)TlU8?$xs9V&9;ucm> zX=jZr!_wRr(rIeh|7;ks5&jN|F5hx`f_!>902p41#_&(NlvI3Lac`$I%4StvN~{I` z?FH&OYx9+msXB(HX#vLnbP%-9llgjNT7MXmxVsUqT@RHI%_g~VES&!do!|w_|Ls3w znS;+5Hznvl>heD~|BnRzM*{yNf&UE&fRo84R#noQOjtjFRfeVOO5!ts;thm?1MU+5 z62(qB=x>^fIbnu8oZtTO?PHI8RuR|a^8f$YR}0(GxNgW&ZmPGl z89&7m2r_`Zj3m8#@}iafxI4$msGYo|NS*)8$EA@K&Qu_%=p#%3O58%5*=AQ3m_l0;_=0OWILSKKV<=rJhZS!oMT0_q4WhoW)l9#ZRs;%xSGay z7cFJq5E4yIbBsVt`K(}M&SwEmXIJGnIH8oRnQ%2Q$;o112^EU=K1|IxG#>rOGneQ9+-fx^Ua#vsM`}ZGo?kx$}En`x{b$FdC!o9 zEK#@cV@X3dv=ohvyEBBg^NYqx*GAVUBKeQ6R7n>YS0~-k$vu;*MczLq>ESH;)b!L5 zH*s6R3WMwp%Cny>{wXj3nvP>J&)n=OUGgS9@AG*HT_#jrAN?2jj;rj`@fW7NBxK7UIY=ed`%ydZnfE)P~T5f8M|*oUthkUk=$1lx09 zF{z~80*}jHj?g&|PWW*h3l0N?yKx3{fTe?C-pK<~CE|J|yb$Ef&^cW+oLuLD5Mb8W4XU5&`&%wEvy~;59B;Goph(LGzEB0f3#{ z`U4(c1puIilhZ%9K^#7NKLbAGTh`p%`uWGoHGZI89QWxB2tpAoIlU3UudDR4l?VOy zPm(CJY|DoN28Nb@$o;=BqeQ8%K0hb~i||V-c*#_)Zu&$DEPB>BxR0NR6F-mNiQkKJ znXja4GefyzrHRY-BU_(2STk#bl#i=JrnYkVSvJ=KAH(nQMqmT!5u`{l_~#;rkghyc zPB@MehnSO)_YH49JlHnpZg7ynVh&l(S#nA<$qYBPy*>qEmUesRTPn55YuP#GQGsdg z+x{dIJ{@IMw$Ex~iA|3&vS{@cn%_tF%&4>m#@?Em*q8i*KkaYjc|YJPoS$o7h9@H9 z>DtM9%3?$w|7z1#R5n0~g5tFLNZ=81nE1)GG^unu=-=}NhZ3lPbrhLjjT{^Qj3s}h zVhF>pt2{03s>#XWqDeqF)qniL_d!U{O+7(7e`@#YL^h_7VZdylFGMJDvV3&5(m!nBk*U-QVgd9t9?)$KdexkID z3c$I|LpDpB{u~cWNk}9^e+*W?{qV^R_wj}B#A&&W7U9oY<-o9$qGfyALTH{MbVQSMQQA-w(wQ4=x zj;VXw_HP*h;1{Oiu=H=Hzo42cw;e(ITIv#)aYT46L#;yJ-mrUO@i}8%YN%sTCeic8 z^9tkL&=XMQc1(bTR9j7F0B}{5b(Qv6O5dVLk+b&I>n|z;o6?2tPEdv}q{w$QwzI8pE%cr)-fK#W}0 zDiqy=-j;%d`~Ksd*lqU8Ixos&&fJxioB`j=T+U|B5iv@>)qOjX6Dnz}R7 zy!F=q25>s*zu{WEZxw{cyUyS=8SHz#XWO@y$fRf)yid^ECgj^MdMhh%ghI<&D?|MT z2yJF3*QeP!q!?;wx17zRWx)s#5Xvfu3E6FcPkl5csPw*XzYv+;0jXS9^zvEhOf#eM zzrib_O68LEarg#bK3N{jOa>{`=6SQMLV26Dt2KY{EIdJ}cXz221=(1YLYVhEX6O%K zyUZ_;Ib&me@KQe9afQjnRR3(KMYYm0kc0J(A~{D{baZj3xmt(ilavjzc-i{6Q(|vk z<+^n&#p7~0Q==23^gCBmO$wirawZpMsg1k&0JHxFo1kUq1aERbFaxN%{_ZCtsC1j2 zTYly6y3h&ixruk$(0^VuRo()v1ehnM=!N6AH(DOGv9+m^pGektxu#~R>Yquc=+?Gh zO6ctE&vMn$ksZDsuQRsCny7w(3bMDQXLns0+#RCRo!XqMxw8L|@~N|_2|oU0bxh`2 z8S%@k(|?D>|9TG#>sUhLe{KH1)&c{nN^znBz`?=6z@fk)p&%e3K(ryDKs79A04fIz z8U`jOIvW-?2|Lb5A#o!z3MEomuZf2(2<{EUEQf zQLU{F?;$xATd7Dv@G=^JH<63EI=^dlH zyH$V}(TyyfaF0^b4DzHNzA;eV9_O5r;R55)=bc=hrHQ(LWs%D7K4*-`r`0jP0TCBG zO;mE}+~xJwM9+Pz9X*{|t;noqlApSEWoA(vM!cuBUxhL7OyO|{s`p~!2bI;jZQ)z> zPH3>N5J;s5v^nE+>_#W6nO1g`4eXj+$+6G~MtnOhzKPd4J--YUT`U3>EvuWZH!?JdKsw`^s_T{rBZa%3Sp@3Kd z@L|PoIp9-!nV>~|K3SmUNkrg|;OxDAdg+}-wBT00B1fG~>7B{T<39wqbZ$(|2DSK{ z`P~*P!zz^NQ@HVx*1fv^+*ih})X2tj3Z`JY!;Z=3|8KHw|Ap*tKw1w)*Sc`wX`9P} zgxh`qspf6qCjZUuhZu&Hr6bJykCIiUzkJogd6QJ}q3 zE{HwgWYg_6sNnT%&{-cLn_09LK83Aa@I5*%#{3H!#A+ykhu(+p45tcjyi&c@^Q}W^ zcNz1Wa@6|PekJ}CF;(R#n-etLbaW54TEsM2@A9*YDE6rvw4yn3js!J%lgn~Z@Vp04 zy}>RSw+*_&G#K=P!}J|-wS*0|J%M7TsG9pd>EvM%o7v*WgU3>Xm6Mz1q`*U#cqVhcQ#GmPQ~_R9~c>=bB`Q4UQ#l=_;B4j z%?P+{(mfcS);z~u3Tms1?a>B|0DtjvGOyKg=jVUGc~1vB$tTG;(z&{>fcdc#m;2K{ znQ*21eD+B2r@eRCtDoU~hhJO*zrYKlVEHbt&zKV8r>K400(L6mq);%y(;kr*x~F2# z&fE1R^iy~VJc-OXXyFXUiyhIs#r;=W=Cx+X{MhJz*{{@DGp+^vkA8-!YcTmc5Ph^!#FNh=9tfXS~U*n6s_MeN5wG7B!YV8Af38(89v`T;Zh{3^j+ zdvQ*yD*cd&N*C%rNBSF(Mu`%qrCaWFq_Kp>ZJ* zG-PuLZ&3}Iw&?UkK zyuvhWF5MrsHV(Z*GK~x4bW?Y~0Tptg+;Fat1JU(4w!PK$4YBKyrP}DqFWoMd1-1%X z?^XMH^xnNkiQ~^dX7*wdN2n9pl+ni)_xMuwDRYPY2j0*}T|Qf2?XanJcZ1x!Gv5tz#!mf=?q-q7Wo9zTgZDe8BnMt7` z&mG?HAxaWo8kdHXleJ5hD)Q9lb-uBVdN+Ju;>Z=JF9CCMqKwIsWSKEd6@OW}_<8)x zurLSH$mj=MGed(K^ZFMUp5tS~E^j}Uy{iBv{w<&xS&@ab99Md%f)+BL!2q1Umf{x+ z*DywMni-Z&n z?SHToRU`&l26vZmDz`UVl{fxG(A>anv_?ZZo@fC=wGM_=@sjqN9o8^}sn1Nzy7_0# zvY4elOk%J4@#r~9 zEV6}uN6l0u{?TChlgEFN?4=>U?E_G%xAxC!EU#g1tG0oV(3VGg=r(ATiBwU_u2RK1 z#f#D=hcHZvS~VO_c==|17Fvwf?YJ@k(3wF{By;s_-X;_yvym$bjq!X?KdhG%ga zF$6rriI>&H<9qwAIR_GQ$IwSjGXUnMqf`Saq2%VrnZhy(R)gm7CajOt402&x7IoXLg#txHpp!DN4Ax*Qbtg|Z@CVpcr(ti409*$A6rAz$jJv-KKyYRRu2oT7!KVgYfK)B zsqO;0*d}dqOyxk@t{6|u7%W7>!4+_o+mEex9fDT3eWV*tR_#}4rPjr91ZCex+LZQv4ph<_Blv~u zd0-K|%e6D@z{UuE11t%$mz6rt#}&)>esS_Y41fDz8vOm1x%>wRstM2O6JEpI+dh@O z1{vi1%RsK7T_*Z9XHhQVwTLXdb4|2n*!;b!Uus4>1-0RKt1`X8f7;2VM-|-~|oDu!%FWtby zLX%MzxKV;W+lLLjGjI@C?p@q)6-QZhWM!KMMc3?Af9*xjOATijU-jwXDF|uL(o#ZJ z6B2I(Jvl_v=!Ymx9e&Me+uqvK-+(Wz#~Ihxff9cvclB`DvS%KPgh7&RXK={-V?f6T z-R~wYiBLI9UH#G5KWm8PW!RtABDJ|TtNUw=DgC4KI8Z{&gA9) zGD^fR@-q2U=o#YE#)qJhX%M*Tp8J~7==#fG;Okiq3_0yGzYN_>4COu!PSH5@$$TDZ zD_1bq&GGF{vR#ose&G13t0T5JDf5|(xe*6!4!PD%toa9td{L9tWBvWfp4CiAj%JV- z`U7Gm_4nG@`x0ViQF|wQt@3{8bM;Oj+w_s>PSe@Pm|nyjmTu4${+?52Mh zMsU|y_IGNl&+!WQj%FZ&VP!TaPZv6I8A4DIN&3_e;T%%epNx^s9k4fBmoUrzS^i(~&MSnWXEiI1`>-uI2b9EK(Ed#jNd9}YZHmvY3Fs0Nq-m+t3cnqYQA zu^-iL`u1#dp?BKo8s2R%KhX$cT}3hG)tz~cAJ$3#X#1@Do}T;jm+IqjpuJj>xVnxI zl*qX0Ff@0=6!k}|`zKu|^9?vPEE!Jh77g0A?*&%5 zyK*G+ZM&sClvM`1x!wcJAm@|%ANkGk4G4JN6DuLPu^+uZO|xa_HwEtfh#Gc- z{-7@2kiS@OKBClPYePYQgJ5y=CGEIst8iddIP4i@CoWF02Q-Q%t3T!B%Aj;r)j3R} zXScS9l!AHNf!29H*Z=N>AP_y1>D~ z^&?dGKOM;o3xg%zy8A^c2A_>OkV6?oCY&$E^(>E2A-953A{v^w2&UA>M^}xxdZSz% z12^V0Kb*RDWx;j2P|+*)(KH)1AbDSAmW$D2^CaE4#>vOG*(k%YuC{@to`s_*Uu#um zga)^U?=Qd7Fphn<^+{qZ8^)440!9>!8T($ypmB4!Pr>^!GJobb05#B}FWYb&>a6b} zxO4C4j`e6vl?~Fqp3e;y79AXhw*v=D^~wtZj-Z_MQtOmFS;^$_IZl@>pYw3h3zQJj zEUM^DErb*5;>V`&FsC^6!49`S=zj5CqWgK!T`Vlg8M{seKKpy^Cv8|=xK+UkpWgsT z>>!Cs-3(o+Fqb>acOMNFZ6yK_3Th` zWf8d}{!rA5uuA249R#{2<>dK!83Qk^QcBF3Bwm7AY7_M`^d@e2ayQYv4WQ^z*6c5M zN5M}_t4lRAR9q^9sL!s_s`E)R23`ipGAAvZMZ*b7EQ0Sq%q6|0#214KG;eI~dK zMSub-5IuBiVLpvvlKnJ_z94r5)u<|tJRqc1tvtz^i$MWAauf9lL?qSxiP`X*aXerd zUQhzR%R+TJeN03i&y>fY%%{VG^g6kJQyCrsMN`fD zvR45JBH)hl_;V!5x9B4AKvLr!{L$(HZegoiKZdgCoq0ozy&p3JqcQJ(HI3ud_X>>^ z9^bhN3$`vD;1eDl(Ci{W>OCj zy{E9384hqSau0B4TrX6=TB47#B`niQaNlIqDgL1|3ZkR8spfh_GlTIPa52SLpK0N- zx9>Oeag3Tv;_~pyT8#T`AjB5|v z>5e~miEv{rfH*aGEE@r(cz<>F$(PyRa+<+v&gC%WqIP0*o|S>9CBQvHSA+K+V__oG z>Plg@Y5O@Fl4IM$V0iZSlqk3030h*n2?~erXJbxX@3y`JHVrY}*_V$Gh}Fn|@|I3sh6P+BZ^{>UV$PDG(Cdtx$h_-U)Ala${k=>AZN%$N+O_&G^o(Fq|oSS$vg% z6sO&xpxsPXK1G-3A+oV&ZiXh6s_lQZfjXlc>f`vQm#Y8JIecenrnQ0Q$NQ=5Xw(Sz~NGFCT(+H z)?Ga>Mo_w*y!Ush6zOm(fr9VZkb7VZVExt4jKw-b#thHkB-t2Zy{^1#ueM}Lf-{EB z07H+jum&^4$zG`VvA1r_uuUt0wl|qRuEnV?n44>jhl~v{e@Wr%35lin@f-3oNNCiY z?bV9Pyij7|WbNQDXMYh}UNpayd-h?!nXkx?_RHFTdx?oz!ImegP$|4U~lL&0UPBP7i zqS)~En|=enas6smxrQ1{ax(o0Gb<8VE8IY~5ZMTHBpaqDgI5k~%2d(oZ=x*!_)v9G zt@lT%-$oLt>=O@cXQpa!RJ|Tz?9TcmT&+2_x?X4WUr28Hj6CMh{#r;hvtyO7%~_U8%-h8nvp&poiuf#5 zVLuJvVsCd}tn@(juNepFERGg9NO9~K{}w&$uA|5+3Vki`m8zhl-LmGUvSA~jEuslr zYYrPB4*mECNl$+kpj=o$nsRep`dU@pT^r|aZ}`D{pniNfApo{#C2VGs z-6$3>%B2r9aNG(bluJ5NZ7Kx4oEL|7JA!s*VXgsdxf&;Q^4UMMk4jO&3$vP+p}|ys zZcn_v?97apiUin{=(>z{_Sev~$Cl}4KjCbKo|Jm;8>Y3SWK7^dH04A8(Y=Y{&{eos z*YGOcpL&w>>o=fy4_8#i_;PbU%JnZFnaG}h9;rWzD$#JJr@>xj!F)Q(Ra_?GwTNkC zoTD_HN(#S+E3#43)wygd{1E~}eDt3dH4xr2RV4Hqke+|@YjGz)$M-LTvOgNHC^OB5 zf8J+dpq}3LH>;ZP6VVHC>OK8Yoy1M|he??tXUKWljZuBIAvBVnT^rMWBHMRes96F&kkO`*g?B;B+a#4(*J zYn`7R-lAtHf5Hb!4AD#+$c+h)I%UQ?6p+yW<+{00;#u}*4*N>p$5-p$^~$|MOw~-P zw7F6}Pu%p@?;-1POsYAO&FGh#NB6Er`OB+z-PHzEk}whAVN4X80fXMO7Or2*vDfPX zl;Dd`+c*RpL|1V&dJNF(g*Ib<*4~Q9l$UX~R#`*kY`D$>&BU)$voogLs@B7v}@piwP zt|&XJYlok60{TOk04I%w)rJX&D7LN|<*PtwPnjHI?@=t2Sjy$6%zuUV$ zi_&K?^k@7(ygqaO3FF^Ei+Ch&B2cn1b&);R8*X%5#Q7xBLz1()Asdk`15y^038nch z@O}6Hvn4YP(Sxx#%-qwE4dcS~4q<1QvXB(1{znT&XiDtD(RF)|TD60nderWX^_VtN z7sZ*nPnw1&NBV&|WaHI#;#j8)0>aVY0e@)D^KCuaZms$h zDYLR^(@0a$-nPDNpT1t|LHGO^cCUV;_=Nv+@lS%y)#_R~xi+M>Mz0{=iV=QZW>^*1 zXI7R@0;^*)k-Gw6`ozCg9jr2_M2Vj;BJH~Z0qo!_==6dmXkUC&@Y9H8JL%5d4V#OJb8~v}5XyDv=X&GupB1RRm2n%m zj(@j{HFFMQs?qUF8fiM2Xg|buB<(WCsujto#@#+@{zMI{8|2KdnY~mOHul^#)H#Lp z8uuaF^*2RHY{HFXm-bh47`dD+P3hr~NW+O+Tw?y2rIr8OwL6K}Z8n}>;Ymig89rJ; z5RwKGz{`G0C@G&>%X-hNIk@a-?qojWmT*p`Sa+{ZAd%&;FO`OoqDH?KO0(EB;LJ(K z#$v=X8wOvr|7$IQ_7u}z5TXU2*;HQiR;5UkR?g- z?Q|Su+lH{$!j@IPQZy?%xF_9Zp3B>^M?zIYs@$O#*5?#AuP^n1UFf_7>4ypn1kzm+ zu7>xp_s0v1%vs&Xi-L+=6|}*r@y-QIj0>B1+=ppXVHyn3I628CI>?kEwp~*)x)I*F zn|C)e#?nG(SfmyTP(uLas&TtZ;Ux0c8c?>eqeU88-{Mt1A#4pqYIIg zW^FW@7Qa$ek~5TX)Z>lcfT#OH8(x=7`u*4Ii~89rqLO<$hZuK zdx;_5%#o_td8Y*rgtD96!5x(hnerDpPGwjg(vsK)9dA*NRo(d?z2uW6cX?Ia@ZMwe zBIoNSERrQNoI>HClp%q^uIuG?4c}jSJ>zi@&M`Y<47}_O8*LT*6zw%MknuwxOUv_i^>}svAZ~ptjf&?gNIM zrW()I z_1LStmu_7$S?E-5;ZGlFgFT-x%Ob~EQ(Q|YlERMAY1V7>#gm86^G#DL_Ocr%&pJ8G z2)vd!WBV8TFdy``vl?NqNthDHZ=!iA1?^_9Ig4-b{-k)<5B{8tg9x>mMv2!0Dqrh} z4uUPC)a$=rnVUiK<>-2t@KGfVZs9`1VI7|!N#$GGJa3p zHZ$!yGGOlFXt22}_|Qy6gU0@a1J!qPqNU?}Vu~R4BFZvD`ou*%dz}|9jd_;4FCuEY z{?k7-fahiU)6kU?&?pLd4i2j$u@4+mBZ>qtD^=AGG3Q-nGG{=8-) zVa+;&SpP`B&Xx?4h$7Xb>|~iP(yiVGcZZ@S+?B;XWOE;9M{~IPcrT93B+eZ7&ZO3U zf=0=nl|ZCdy{$Jtr%bEakWO#ZHTHnJcvd6Y?l*vJ2iLc}37hxD2R*}BBM`Ofcm#-i zA59v1a%4*~HTswEA9nWsI@kazc_r#yy*$K%EE&jgBd9p;`>7x$ef{WG zk7GsE0|aXSs;~Yj13P>mH2o9fIjv=Aw2#R}*eJYYDyA?fS1^h2z;Q&n#!!+P=B`1skI!RMitk?44a7r0e;s{yjmSQKlNuo-SdGq+4O>?obxAflXU?O<3!; zyXdm{Dm&XXG!@P?HxG2I->lmc7)UbrcDxRMZ{9B2JV7v7{;!k%)dUd5fE${)93Xg&@iRXCEc9^&=bh|O=4jR5@HCreZa&l3 zA_ZGFSynqnnOxJnMVo-W+r&d$yL>Q+h9)8TuhKETU^ik@htz6>f2yIv=#S`Yb)yps zCS;9Ep9%PS;Pq;sT|Hi{21*WlM7c(LE$YhoI2FsVbL80n|E#ysc7@Yf%S=~nShyoS zzMiFd7cXMxLP{p-n2Oq3(4Ag>)0tpVTg20r&rH?4bhu>4xBT1C-#E7o&h%|b09FHf zMd-Zmn8ig$`@u4H+Q+MWNuD;`2@#hG&(j} zj791Icrqq;G{W)?SQt9unCl3(dzBcC`F0fY&Lo^B8}M2^>!arIEdX;Kgu&Gi50;RU zTGH5N1`*VH)KIoq&>0M=6=f`jnSs_)ky(tIB6w^ZyLu8OQE0Zvo-e$>y1dnB+#=2F zk)yLT47(pa#?Z)pQg#P;qs}?4rJA#BnNjKx3{tjZh1oay2jyy%YdrzjdD;ul6D9gm z{Cm6'vrTm$$%CvZbWk*OckE#!5|`_!6fR`uNmIhF@Az&(tU!!$bR*^GK<0P({g zl_oV{92f~4eX$0mc+si9Of(WVjg`vK5f{8t8v(ZY*X_xXKe0s%-ZO(vy~%=3y@7#) zLms#n|e(XkSP+d z9D)UN5Z0km`vik=64p8JZmqk-wKaR$sMMms1;ijrU>@^OC9r=*f&au%+=vcrM_@zk z>>4#;*0LsZU>I^Vu9B@e4_n)+ag~1WCB2RZ{o#~l#%7lNM$ncGpQlM>24d3$vGfS* z*zCzPXbuaePNz)k(7Sz@!4Cf$f}vP*N#vm4x)Ll=;ZkxJ4zo8b)k!1#&1h%bJ})Yq zna~Xvp6RodPziciCbG|W_M(!ZLca=Q(`FIybp#2o<^Cmc0^CDQqH4!CAJGFs8v-%Y z+~s`h~=UCwcFzPg>;q^P@`+%F2jxe)7b-HrY8?;rydtOMYa> zSti7#LOQF!#EW-j|6^>PnC+6qcbVG{1<$V$qufiJkz+hH{EJ`TYQI{+jB>AdL=HUD z7i*(ZWDUp~+_w{%SbD%}%-Y&MS$}#=?^J_y3cKLEf2zyWx{|NvaOYDGB3Xcxo=d@9 zE3?{G@omsBHt5-IV@on)5oG|!&^BZ7@`Z$IkK976CVVUh+(3%wo0DYgC#}yP0 z^au+QN~zoctlgncLvWhnagyBOZ?(7J<#4DhdFKd>6EjXSBExnwXPWLP<-$TKS+kr`da>JvDi<^ zTbB*W9AZfBhzyd}1;LhU)mDQV?_u$sqY=O^%leb&*H1GPrqwS4hgBB#FI4tdpAQXI z8VS3cxo3L|uHU<`ja>;p+i*zt&A7wK98LBF>HP-0VR2KYQN}i!9Jzlsp@(iV;4~+_ z(QClYn~@YtO4Ict{|McGSvkmJp&{gGQXyu%##&v(*=eeq8g}R|{Os)wp?Rge^?0fS0`bwWtP>Y$pr4eA;h! zio|24GE%-7LpQx(Td<5o;_Kv@?~(CqJ|!8ll-`y{t({A+%DQHvRgjF1GqK9LM~LH9 z-K-dTuil`D{YxzRWO5N{OxjAfxX}v3>4=oyDt-NirvMRoT9P4%Z&lmN%24~fq(p+X zI9S4Rr7B+w273_Sc_Ed%3X7L-D_~}5hkIF!ev|lxXjr*pshsvf1g`<2VPYy@$CLmX zXxb7Jl)b$pn2E55vwqRS--;Y`1{jJ8(Jh!WtjQW>cCW8Ge=+uu$5mu@QtQTpL5cx{Cz9 zx)Ft07$R8lX_pzV4`4Di*2ws0M+~n@D;yYj1zeO)?CL%FNgnzz59yqYYz3w}Ai`z6 zd6w}=)1hFE5F=jX3Anv07BD;2$G#ni4%c^tC1$~a4@m99q*n~4M$3K0$}B+n0fT@- z*S_umS7J;!sk{6oKCVgRl22&4@H+xs{0xE>?~X)TH>#HBT-T6y&ZXVs`^9_!{;FIc z`0Ho;XLpS|Ax6a%KdSGxYc!oz&PWX2wfbQlLiPRhMHO*1SvcNrr%Vc9 zwoQD}98q4|VGou4UH2(k8L5SxQT?4nVydZ0xU7v!!k7R{S;6|Qgc}IG#70Bi7~xJY zO|DuFeBP5x!)19lZf>ukzSfZ6fNW0f&td^9v+qcfYiBG$w{$O8; z=V4&`*2PEI<;H`xt9M?5lp~l%#3hJ9H2o-WndS{ zw=$aHo6YB#gyCx#>k^%&Bvr3-PGH_?-9Qnq9araKNm#=IjJmVt@XJ z)NO>4M$57iP%Bq(A4{F^W*@P-6a&@mx0OoIMkm1qWQaK*K974NXEd%=^H=bPL>rgv z!x!aj=(LL`D2tJIC82GFNoS~}j1P%TMqXKv6xw@|6$Aj0jyaUt9p<%?;IAoc_@<|2 zIwBvJmmn_b<=yNHzpHS5{)Lpx&NNmfeleLY;kq0^r&tghu;QyQf2s4UI7zo{Q=DT# z46!2hW5?Vf4JW|X;_s0m+FM#pr+(EGCtAU>)UZZi&(wy=kAE@2D)ztm2wP%;QRvu_ z%tQvga$%~K7Utbl_|rCfBubJ~4APIm+A~)sCUc_0c(Ou*5}e`}UEFsM2pQmkHe9*{ zwiwqwgmvA#@e&%~M{R7ZeP3{DLoL)t6hh=vj|=cJO|)c<z%`-bs~qwzzI;|8zDn+TC3&<_yb7sgggSyZJh;{w;gF9cHe%$0;FAX;?eD>S0g zCcrT5bXP?c2{c)mZi=w2)z(WL_o{vaIJ%&G(lFtZSl~wMixeB{(1Fa?WRA3f%BoOt z>q!GbTZ|?4a<$14>kPqOrUp@Ircr7Hg|o_n=wxRl!R8kMa z=FU)`NJ+F|#plBda2`cnWBWL)G+fCS59vepj_2l^u0;c#q@}|OSCIuv6hWH;)}Pf^ zmbXS%c|fgHeDhTj{;Rt1c`MEi(J$7fUWpBw9I0a}!lE;^=`U^fxN8>sUyP_MljR+_ zjnes~bysi|89N*fF9bwNSmLfIlD3i>jB!e;m9E>h$nL^-5sgCkfE({sm|3Rro@%-& zMkxA5H?#4gjCQROCRoJHUOoTth&K6gh)4KFzFq-uHY8DG0^6T7X|E)Gge8T4)*M#@Pm;U^ zhni%nlcn`}gJ!z`IV?fHbyebZG>37((!W#9o`bM{>xL9&Pr-mn>6&FT^%|ca1Ah{R z6ngBCqX7LWVDN&SDeGq9oT^y}RXnmTiget71=Iv0;={OhA z>0e{jBSEGcobG=6HVI^p6(89 z%MTyBd}MuQK6H_*R(fP(?`@EtM~bSTtLyv|@|v-6`$a30)uDtW4%zwOuuU!J{VY8+4`@^8d3Z^C-`qvx z$b*(o+~}7~VbJD5+q`{f_Nn!O&9V57bsZ-~DoiZ%0*BFzd(&E>Emfm5n{Mc@tEkHM z_9$6?p>1u61s)`feh1VO+70@!8xgPBn!(5D-vAkN(>3HKIaa8zy3xyR+WIeS1oO+0 zp}`l_u*%Icko5O=AHoo7WDuTf26>IrMLGRC8E+Ze-dbTUPrrrxt>=E;miG9h>P&LP zaVQone020__gJEeoTc4{ApV)RA~fg|&?$@{A`4hh-4)TV)TAZssNEsCZ>4Q!O!xN1 zBIiW#zTm^7=5sS${-Ap{Hh7neC;rWlWPW^?oc}Y~Td_DNv#$ioQ`G|61U~Y8XRQL6 z8H{Z+>t_Qmt20*5)lY_KPsAYC*Lm%UPa;on68{3XrcvZos@fD5=MWkuIiI}X)u1Kj ze&E%?_=AJSD_393+GL@zODucs?hZZ#7bh|)uf~l9QTLRT4yB1v^UK{%&v<>@Q=E6$ z&-t$@J({9{_^;tTUd}xYR=+A`%$?9!X*l)EQ!pN2E<}{};HC3~I-vKn=l3OUw<{** z+*@K5t=Tz;6PS7Dz`I}6=^x9ogeg~ z&sZPhP>SE#O3D=vIA$}3ZU4qfk|2PkcYl@r>>^RI>$CnMf-a!yPgB^`jmqc>JUK+6 z)uH-S-z2Ga1@zSnrP;!ZZm7;h6ZuMpxYK+UhrqCs6|l1#$A#n=_A>TP)iy;}VLpxE z=**&g9tWW`Bz~<=tTHpnhpV|Lt@bv!yok}i#Kg6=vkVnpApU5pqJ#7}spi}Su)S=R zXh>QHHg>hfU0Picc{o=cz)0phoFB{X@$%w(ZFDAiX0J736V)zp%SHfvb%i>}M%yT2 zdk<`Gz}$Z^Lq}YdnG-jlRXmLb8GmUX55VMx3s22o#e?xbMK#K`B(7?Px0hs}-qXP^ ziIw=$PytGykX5){gUVOAz#Grdox|{Ftv3R*ajb~UiN?NJ*Pc2cAw*=k`qb2(63QcZ zr6)`~c7N|}1Fc|r;rhaMM@#hRq7O5=B+3kl5(J}! zL^paD45JG{lpqYFmmo!qp23U~C0g{BAYm}N=p_k)gh=irzpV9r_x^GJI%l1=_u6}% z{k-q{?C0GDZ{rUgEE^II<5;Dz#8P^7Zw<&b_F?JnL~DgEWBwoZ>gk#SCH7^AFZucM zX@pTL2gS-LNaffKqXS_Zs6ym42z$z^!awWutirRt$SC=1}} z2@D+k>DGKedSCm@KK`QBeM@`}@qB^x)^3#E&9}|p4cm`oZ}7e}7$~Dpd6^yG#TYfn ztz#z23r3itHoLZmkklWpj-|_M4J26VD&(QOEuu}Nj~oAp6IWfJAi^wwThd-$iCD{@%s z{Iw776zUbe^d!T&(a?fDukRkx9BO(r-~lxs_fhwGBD4e!5+LY?Le(D#Wpinn3cR|o@vDGGQ+O4h;{GQ{0Zutx^Ko5uK6QWOh+BxxEp=OfX7*;{{#a-Aa6?16o^r=eLi zCBL)>og-g6^3L7L`M@nHkdw}}TpUWKGg+z+sKHFsEfE63&aN~7&prJ6xhah%B#Vbv zpd&ua;~z!$^EZZ9AhBOqZBMh~YK&ddgL&+Yh}00rL2#bcpcA6F%&$5u!~meq0sO%6 z_KmGTUS!dvM5~>rdY-mhmsmf!be)e{jJAhJRxG7YhILF#943)mY@1taa_kjjDF<{- z>n}zb3~zbw+^izsO>D0@Z+mc|c^!A;JkI6TP^LBeY73R~wzb|d5IJt>_{sTOCpe|k zfJU@Of4&6GbT4fq@9{O@_WuSmqEt271DSCp)k2WZ=`jr>rl&hGPGz4-46kH&6J!_q z=2hOc*v)@29AW79772C4O}O$IOvd-|u0hg99UVSX`*^Tza9{d!4Ux%T>P0y;mH{|@z`KsC+@*RHW7f7( zm-bS4*b~fu^PTjY zfll)KeLPf^ECtI{gHAoOjjOk8`ooK5Y)Ms%QPl0lR0RRcM6zFuN6vXlAn`SqqPu`Fcr(~wWjNpRJ?i-=D&)$FUT0_4R{4=!&?5-Eh0id}yS%A3)YYrgBvh0^9{v#? zmcLTKDz^S6dYQaX2gAdz)qaO7IYZt}aN1@zH94brp#OiyEV)Q7 z7l=nP1N@HNAISj);W8%9Gll~hrQPjNO2;BdEP#90`W`Y_QSWK8d@#R6xT$k~wW>6f;!d~YlXd=O$6KB5Hsb?vm;K{8ps|Pw zN;}v%f5_m4&U9=Ls#`kyT2`)GL+@*WbULL#4)7Z4C-xMl@K$n>l$ z0d%r^(W5~p_Bb`oOXBZ`H5^6OM8!#*-ir_uifNw%uwf`o=|$Y4CJH5A$3%rJWy1pj z!Y2NK*Uflz<;fc@=ps)m-RB~u)z~E>mW|66{yo?_Q!tSyoOq${F*g`{O4|A9lsl}` zBkH9t#Mftr4@IzmDy@pzdk4 z*i3UvxVZUxNkO5nwwBlIAEaZF=$KVX)K#yIYM0F16_bbF=+WxSC^#$dhUP^kavh{? zezT8n*vM^ZO~Bk!buleYdYb;|t@;2J@}XeSqETJKduw}lh*x>ZlLIni=~$VCq7EwU z9XP#aV$U#_IDOpzj{I$*73x+Lz9WPcrYY>p~uVtH? z275E`-VFVc+Nf^BJ$AJ1S8r3QfxQOjnC59P)lE_T3ht`}e>U1ajve?kst?p5J#RfX zxq%Mdf*g_<&sHQu`(2WT4wJYZJ@aHo<&g*0`gz;d4j1p^0(|=xemuoAogq)}b>&mN zXjH~kDQzTrTy3)#?&|C${s>g=3QFd4n1}|c2Rrvj1{&5e0ceSX#~xd%SyRv1QShV~ z^QD5>+7}QXrP@X@5&SFO$`35XGQ>%-93j0qa-|*5W_!Pe2uiaPI@2wdOU1hYtg`|f zy(7RMfVbPc>xKwVRpAR9P4JCIoa;g|Vi0 za;)^taSU&Tgtct8Ga?BcgV!0uu-{iOq~=s06|>JOl3`E&ucdlt4qf_kf6`!<(Qxb0 zDFCa+V4=K1^i^0qjtBB1vj}`vRCKpRg7nC9PK4H38@5+V%u`<7Wrvv)?$+XsdKgFg zdX8y(X>2*=09YvL)TC6krRk#iX?|Q|gEQ9yqC&+7Qzb5H5Dcivo7PCVFrS@l2?jnb zl%7!1a<&NPF(dO=@`jJ`p<|lF1BmWXfM!t#_7q0#0&~AF>}V6g=@eSlnS%4!-vDl> z{#u#Sd|u9H6b~Awif4q-&)%@vp%*ND;-z{MJAFAH z?MY(Gje`@~ER3?7ykz;Bk`^+>|#&_~A&8>0REW1~?q$-v=ILR`U0fG{^?s8VLoI zO44SwY4u=^u;L3TgS>GUYmXkg7;2Tknl9FRmAwNM)cNNhe(=~Bo)aJHDOh(RQ!+&L zsvhU#@1B=pO@0GB>Pb}J^Ic-jimj^uQMgk3* zjU7+z?8Y@r_u;4G;|JNUM%&xaUs-p~i)j_WbNLGHx^eCAN8|WH60-!%uEXL8EN(vd zx=&@rzf9~x%z46@vC-c>U$-^)7`tZ3OYHc1X}dZc_Vh%}u3YW*jzuMfW$A5P>Gr%? z1>xy-%Ilq56bze6XY<;drjh{{2r(ou_F63nf$hO58sTKv8)hQIz-7T>ltu1^q)M1- zHxkJB;y-)Pi`OP)7B4F%)O;%qJS&|*I;OAu>~3#P577!;`;6s>Qx@mGe{9kaU+8oD zelRV@bx@u?susLo$9F8}z(c?a{CW|Qq5tFKz-*Y^sRC(76hZZeVKeXLk54xo%0gWO zT-2^1cFov%Wir3n9{s##1Tki0DcIXF7XOkMhveN$S2eOr+ULk(rLO>9bQJS}>8=L^ zIKT-U?$L4H&uACzZFP(ciJ@#e&V4bs)k1k$nW0BSPqu7R+s4vnirOdjQm`{l)=Pu} zS)J&tgGYa)+GisM@_sBy>;8ag`Q55_xni6vqH_#eLyAmh(RfOWebXmvM>)8#Iko5l zMftkzOx=7-hHNC&s0a1}{8LZQ=lj#|vf-?_twT}R4#^p`XSYnNeknobu+O8j&0WFA z0K}7a!P^16sN=OEQh!U+b^&;j!DebQ@)!k5T%z`$D7m%$T#I;D+MKuDUOch3#sf;L z2WV!75{o6&mtP*(_)Ax*$K{1g#*f>Of{UI}9xVrt*DNBSL`>_gy!EkL`3%^^&71>B z5Kfjkn@yAX)PM~9gX5{DR-@ zii{Va-UoNhAXm$HCZ1ueK3Xa3t)4ksZT`Wu2H5jMwi2Ul{zb+wT*?^tniAfqvl?xi zLx~q=A!e&e^XFzTaNC{8~jRj-UNdQiDE-yt1nbQnac zTBynRBaInhW3#=7`N_2)vqIANNpo%wvNYW`K~+t%0B<_i@LIB+r2J{oiwg}Km>T7q z+Ifo7nGxM^-S%ZHq>{30^(A?8(hA$vkxs^sI&#xK84;qMBpH!2vgMloZ5eQ8p8=4b ziR+T&v*JzbtHN@))1~S0rFtjF3DY%_ywzkJugM^2g3B5McsK|8oVi%{>Tn67OC*9> zipdsVs>ya~1Z6>S%Ws}U;x1i(yrjif=W@1Gozv?3b0`Ac!QG?FDvcNO(y5y4`3>+T zD0&z#tp$T^Of1JAiLV0Z%mqh0?uDg@FMb`NBlt!-&Y z2ba+Yq`>FyLtimSUFqS|n~2v6QX0m;5jP5N3I7)K5x9>Q8RpJJn)F=s=4VT>eFL8x zt82-#-YY<@Yb~>rcv&0g%9X$;ts2^}%r`m*g~_NsEKHQvwA8z3iLfntoVsIO{(`7l zh0KtS4_O^QD>ZmQAU+Mk^2O%Bw%it?31{rH5c zq*$M%D}Q9Ab(it2-vHr1;z5c_+X_ho6WAp`(D2CI_5N;D2fAQFVAvVVf%;AQ!QNN6 zp9fEfaol$(^vghyS8t=Zh83uzv4r)q1s_5q^ycBi`qzn7V7pf)h{VQ$bMx6h&M5HS z7jEUl`xdhHG$pMS9w1(9){G0fyJE+xSDTaqfMmx29wcHlLlOs;!1rVqvlB|+yy0*j zg3nDo&%ArnZrLvP>+I?wPmjk#qyvk_z|JYEWA~%gD&4AMzjaC6AYw7XDtrv&mT@PH zAu5R0q;)wVJ$337q_(ue?8?^{H$P*fZ8;%1h=Qt57fnd9IGIc$S;kuG3p*-XK?jQe zIfJx~#cI4ReFbPcLM>jQZHFnN^0l$)97T~xhUHZu+o<#*_YD-RDNvwmyHOthRra|y zJ0!rM<3bD5Tfhz*1LpnbP=GPy$L3aTKfj1*0Aj{0TBK3?H(==Qcfz@xAC*3aXjR24 zhD@uNP#hI62GOum2rBWW7D57qzM>9?=l=w(7JWNG3`Tdpghu+S@DXFTo)wLnrK$He z2y4FOxEuPOEoF=#gy-6bfWhCF_ZzKYGy9$FU>q(+1j+G-LR0#W`XF?e3Ul7VByR3c z-r7%FVP$YUHTgz>YZ{}{KKJ}(sq5L@_+2^^h_9zkrVni!1OGyQdVoaU6wLktwI zh?g6J)1E;nc$Sb-fn@niO#*73`pQ5xU#bIJiSb*|9ZyE8ecPPUvFV>{M~;F{Wk*w|JT&)_OYex0m5FfdBai^A5C-tQJ& zQX2B?!e}~U=!$=dIv~q)?u2;}+$wy)jaxTH3nG}V_})NV81HXYTnF>Hpf3kVg0XON zs%XBT821&kG!EfAlVfHs&i7-t=|fEkYfg!$c)8RE6QT^9JOfe@NpKngM(%OEYco4d zin`>};z#9M-FbSDIILPj=Ngs{tlRG9WJ=uB9+;at8$O+I71nYX3aRdya9l~V%o;y0 z;umgWp`Ckr@7ONQvFOvas!^wjP@|yCdi*C*O{LXw9d$#sB^8Ge-=5nUdV3U)FtjcV z)Rt0eb=SNdlQlJ2q;(Xef}Yt6QAlEcBv8`Pm*wCX(~Ar1IBYf4O(X%}M0B+p4oFxV zeOS_Tg6gif%{RbMN$(F|8h%b=aZ+SiJxbm8LJK7}?3Fw_T(94`tB z^&By(m$HK6O>d}E7*@TDZ0)E*BF@*&5f{pDSZR{>b`!)@IzFJ5_z6&!er`I6avZm` z0aza&)mc8>@K{D}s6-t&M>c81nSIh)d3|?@u+gOXT;4e^2r%Em3Q47?tPvM#g#J4C z)xsu=6U<7AVPM)cIEj~cJ<_cj>OB*302exLlI_PG;Nts4xD}UzLk+-m-qm%1USQq>+j~9hXfFGqR@eBQIE-Qk0h3 zzj=}!Z4?G%Crf-U{*+vO!c3bZnMao^@q&XXg8hFwY^PandgL$Q>Ui?4y(~H9lXby$ zI*AOpEAC>A9%uTBphXs8y<@?CY87T@VMg2f+5r(T_h-S}NJ|uQUP0IRY!2hTP3j9; zksp zwl6cgYOlRs&%$AejO=MF>IBO>$&z}^E(}go)!J7axxwzTF**u29-lFm@`^2@xnHV- z?*=&vO(;#2gt${XTrv_ke~Wjy4OB)F<3xBq3lL&xR{SccqG2;v5dUbt@UH0%Lgt8l??TR*@Jo#xXG;P45w?x*n zL;mE{b264(f_if{Obgnad;at@J11qnqMx$WZY)k(5=;5{F*Wr%GLnG2hlxK@1BcwBD;Zh7S58=QwtX9J4z`CT`V^Ea$`8>Ol%r8Z*%vA zF{;*y)1;YD9ywU`p@+Y4O}p7kG7wKbr@xjvH^s)HtUi$fm3o|6zS%#@>j?`0iw^~~PC?jNH5 zZZ0VPZ>r_-WyM|vN>4K$+mud8rshU&((_{F8XR(Bo}?)K ziPDC<-$26dhixJ!yCgViwYGqn)MTKCn(I9yUmrS}x#fTb!>}^wSlsmU^WrKsIJ1va_67 zxRKlyAg#J;Mt~Pfc=rssFWhQ&Ihr!6cClmDOjeeEPlgXx;%Jm-y5cn?)xBrbek2Gu zs2(G6)w(V<2`Hy?uk|w)I=C!7R;3>4l%4Uj&dRcwxf>D<(|M z4AXH{g%fet{d3mIJj3^kvCK7LRfvHPqP0iS-6R@X4}8b#SxQjOW+9!59q4w)w&jP`Z!d_~R=iR22hLhQW8KDkTnN7TQ7u8#*erO>l}N=;-xTo0 z7!p~!FMakl@*NNNse9EIfeC}aZsGRX9cY}KOmdNB%v$?L>zvI**LYKVW55n@I*aO-;->r9O?a-IWz4lWX7;zyU z*=L1Q&olmH_ep6&lE9eI@WE-H_E7$qg{(%n*f5Od1M~^CD879Ax^Kign}4~dl<>B$ z){*?1nVJ_DFTy4h$&N2=3y6r#jM~NzDTMOA;eV}wHRro6CVU5xL{MNL9Z2cyyZFN* zmo-Z{e$ENK+Wm7l`08k!mzP9He!?XQkm&iF#SV>lxDYdoJO^c_oom<*i5}VP7upsE z6-+|~qN~ye=pK|A{g?d2sgpD*aW`ii*{v>1U88|FRCJJ$kIWOq$DJ0k*Qjfh+lVUv z7*5KA?^=PmpV`9m^qtE^Dx*d6RvqM$-;z&8kw1}zydx!2sY16T|I1L_N!9wEWGixw zsj9YXyRF!>&@AhiNlRKd>mJ2s6A>cqzNF3sasxZ5bdDwyP+T|U>G6JL}&O}riQNon_aWq9=4XR@l*S#$Jy@CRF1V~!V>9s-D) zp^Cjq-~{Al9wsehRxdRk5Ovpsomm4Cwjw^8bKK0nZqbfzoVHUs?V9A$Wtzz%Z0VCi z5txT;)<#Z*TlVDCGS_xnKAeXqo_i$10A34Pm!`+yxO?lgM-15`3Qb+5J(0Q-F56ZB zfD~Yw>K_Zm{9C5Exlt2qq5E*zC@036>&iEQn(~Qbn1WcR>|zdH6GlYAUIv(RePR`H z8IkZghg5-NndDPWqJwORB^cHOGBTD7bQPOUczc2P*7F~H7?|%}s&>zHS6-52 z+0R_NI;m1ffW$hwQNO&l#}=!3>Ak)-mG%$F>l7$%P30bT>R%Qf^B-k=6jJn~{IqND z=EvKWePWR^zlz>!NX89uO|VcotRy~xXBpB6fEY6hz24f`*flCJRwr}8ft1V`GM*d6 zuFd}Xs#gJ@7#X@{uBvapH*i?a&Vq9q>aagT33ZJasWpwqqO85H7N$=l0)&p|P==L- zR1^;6alJ?{i__TtC80)nrE`Cm{me2l$_yUcJTj5V_!MB@F0*$4hV%tmrvwB~Tagw( zaEzEy#U0zjHaZU!vPPmZ4Z*r$YwC^xx0VR&T9K~;zox^fjpF-0<~AQmUNA=XFwYuEr8m@p=pH#gm>W^7O*4(T%|c1rB9`@e88;9RPu+4*GOIuH st;PA@fa%dcH)1jWubZ;|o@$Kyd$KS&8JQ|T_r?FG1nH8kza?h=5935d*Z=?k literal 0 HcmV?d00001 From c051c593b455862f8652ab02dc2728a4406054a0 Mon Sep 17 00:00:00 2001 From: Waleed Date: Mon, 10 Aug 2026 08:48:32 -0700 Subject: [PATCH 2/7] fix(files): stop validating .jsonl as single-document JSON (#6490) --- .../files/components/file-viewer/text-editor.tsx | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/text-editor.tsx b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/text-editor.tsx index 3742e0f5cb5..2c65b3f106a 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/text-editor.tsx +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/text-editor.tsx @@ -264,7 +264,12 @@ const MONACO_LANGUAGE_BY_EXTENSION: Partial> = { graphql: 'graphql', gql: 'graphql', json: 'json', - jsonl: 'json', + /** + * Not `json`: JSON Lines holds one value per line, which Monaco's + * single-document parser flags as invalid. Validation is global + * (`jsonDefaults`), so opting JSONL out is the only per-file lever. + */ + jsonl: 'plaintext', yaml: 'yaml', yml: 'yaml', toml: 'toml', From 156ee3ebf2be23272ee7d3dd8d066d620dceb670 Mon Sep 17 00:00:00 2001 From: Vikhyath Mondreti Date: Mon, 10 Aug 2026 09:26:19 -0700 Subject: [PATCH 3/7] fix(provenance): feature flagged, inexact sidecars (#6491) --- .../self-hosting/environment-variables.mdx | 1 + apps/sim/app/api/knowledge/search/route.ts | 3 +- apps/sim/app/api/memory/secret-provenance.ts | 2 +- .../executor/handlers/agent/memory.test.ts | 4 +- apps/sim/executor/handlers/agent/memory.ts | 49 ++++-- ...resolved-secret-content-projection.test.ts | 75 ++++++++- .../resolved-secret-match-policy.test.ts | 17 ++ .../utils/resolved-secret-match-policy.ts | 31 ++++ .../executor/utils/resolved-secret-matcher.ts | 8 +- .../resolved-secret-trace-registry.test.ts | 150 +++++++++++++++--- .../utils/resolved-secret-trace-registry.ts | 123 ++++++++------ .../tools/resolved-secret-result.test.ts | 10 +- .../lib/copilot/request/tools/tables.test.ts | 8 +- apps/sim/lib/core/config/env.ts | 1 + ...able-secret-provenance-enforcement.test.ts | 92 +++++++++++ .../durable-secret-provenance-enforcement.ts | 106 +++++++++++++ .../durable-secret-provenance.test.ts | 67 +++++++- .../execution/durable-secret-provenance.ts | 28 +++- apps/sim/lib/knowledge/secret-provenance.ts | 19 ++- .../lib/table/rows/secret-provenance.test.ts | 72 ++++++++- apps/sim/lib/table/rows/secret-provenance.ts | 25 ++- apps/sim/providers/runtime-context.test.ts | 2 +- 22 files changed, 787 insertions(+), 106 deletions(-) create mode 100644 apps/sim/lib/execution/durable-secret-provenance-enforcement.test.ts create mode 100644 apps/sim/lib/execution/durable-secret-provenance-enforcement.ts diff --git a/apps/docs/content/docs/en/platform/self-hosting/environment-variables.mdx b/apps/docs/content/docs/en/platform/self-hosting/environment-variables.mdx index 3835925ca00..7e52d9225b7 100644 --- a/apps/docs/content/docs/en/platform/self-hosting/environment-variables.mdx +++ b/apps/docs/content/docs/en/platform/self-hosting/environment-variables.mdx @@ -181,6 +181,7 @@ See [Observability](/platform/self-hosting/observability). | `NEXT_PUBLIC_CHAT_DISABLED` | Set to `true` to hide the Chat module: the workspace lands on your first workflow, with no chats list, scheduled tasks, or editor Chat panel. Chat is shown when unset; `bun run setup` sets it for you if you skip the chat key | | `PII_REDACTION` | Redact PII from workflow logs via Data Retention rules; requires the PII service and a cluster-reachable `INTERNAL_API_BASE_URL` | | `PII_GRANULAR_REDACTION` | Additionally expose the execution-altering redaction stages | +| `DURABLE_SECRET_PROVENANCE_ENFORCED_SURFACES` | Durable stores where a value whose secret provenance was never recorded fails the run instead of logging a warning. `all`, or a comma-separated subset of `memory`, `table-row`, `knowledge`. Unset (nothing enforced) by default | | `ADMIN_API_KEY` | Admin API key for GitOps operations and organization provisioning | ## Enterprise Features diff --git a/apps/sim/app/api/knowledge/search/route.ts b/apps/sim/app/api/knowledge/search/route.ts index c4c412df787..e0530f6b772 100644 --- a/apps/sim/app/api/knowledge/search/route.ts +++ b/apps/sim/app/api/knowledge/search/route.ts @@ -604,7 +604,8 @@ export const POST = withRouteHandler(async (request: NextRequest) => { !(await importDurableSecretProvenance( resultSecretRegistry, metadata.provenance, - renderedMetadata + renderedMetadata, + 'knowledge' )) ) { resultSecretRegistry.markIncomplete() diff --git a/apps/sim/app/api/memory/secret-provenance.ts b/apps/sim/app/api/memory/secret-provenance.ts index dc6ea8f7368..cd14e84b90d 100644 --- a/apps/sim/app/api/memory/secret-provenance.ts +++ b/apps/sim/app/api/memory/secret-provenance.ts @@ -137,7 +137,7 @@ export async function createMemoryResponse(options: { status: sidecar?.status ?? null, entries: sidecar?.entries, }) - await importDurableSecretProvenance(registry, provenance, record.data) + await importDurableSecretProvenance(registry, provenance, record.data, 'memory') } } } diff --git a/apps/sim/executor/handlers/agent/memory.test.ts b/apps/sim/executor/handlers/agent/memory.test.ts index 1c127b925ce..a345073fc86 100644 --- a/apps/sim/executor/handlers/agent/memory.test.ts +++ b/apps/sim/executor/handlers/agent/memory.test.ts @@ -337,8 +337,8 @@ describe('Memory', () => { expect(result.content).toBe('foreign-secret') }) - it.each(['123', 'true'])( - 'projects low-entropy secret %s only in model text and arguments', + it.each(['123'])( + 'projects short secret %s only in model text and arguments', async (secret) => { const registry = new ResolvedSecretTraceRegistry([ { name: 'TOKEN', plaintext: secret, encryptedValue: 'ciphertext' }, diff --git a/apps/sim/executor/handlers/agent/memory.ts b/apps/sim/executor/handlers/agent/memory.ts index 13523c742ce..b33d86b0b94 100644 --- a/apps/sim/executor/handlers/agent/memory.ts +++ b/apps/sim/executor/handlers/agent/memory.ts @@ -11,6 +11,10 @@ import { importDurableSecretProvenance, mergeDurableSecretProvenance, } from '@/lib/execution/durable-secret-provenance' +import { + isDurableSecretProvenanceEnforced, + reportUnrecordedDurableProvenance, +} from '@/lib/execution/durable-secret-provenance-enforcement' import { redactObjectStrings } from '@/lib/logs/execution/pii-redaction' import { readBoundMemorySecretProvenance, @@ -75,16 +79,32 @@ export class Memory { stored.provenance, messages ) - if ( - selectedProvenance.status === 'unknown' || - (selectedProvenance.entries.length > 0 && !ctx.resolvedSecretTraceRegistry) || - (ctx.resolvedSecretTraceRegistry && - !(await importDurableSecretProvenance( - ctx.resolvedSecretTraceRegistry, - selectedProvenance, - messages - ))) - ) { + /** + * Unrecorded provenance is checked through the same policy the shared import uses, so stored + * memory written by a run that could not vouch does not permanently refuse every later turn. + */ + let refuseStoredProvenance: boolean + if (selectedProvenance.status === 'unknown') { + refuseStoredProvenance = isDurableSecretProvenanceEnforced('memory') + if (!refuseStoredProvenance) { + reportUnrecordedDurableProvenance({ + surface: 'memory', + cause: 'stored-memory-provenance-unknown', + ...(ctx.workspaceId ? { workspaceId: ctx.workspaceId } : {}), + }) + } + } else { + refuseStoredProvenance = + (selectedProvenance.entries.length > 0 && !ctx.resolvedSecretTraceRegistry) || + (ctx.resolvedSecretTraceRegistry !== undefined && + !(await importDurableSecretProvenance( + ctx.resolvedSecretTraceRegistry, + selectedProvenance, + messages, + 'memory' + ))) + } + if (refuseStoredProvenance) { refuseResolvedSecretProjection({ site: 'memory.storedProvenanceImport', message: MEMORY_CONTENT_REFUSAL, @@ -102,7 +122,14 @@ export class Memory { [], ctx.resolvedSecretTraceRegistry?.exportProvenance().scope ) - if (!(await importDurableSecretProvenance(modelRegistry, messageProvenance, message))) { + if ( + !(await importDurableSecretProvenance( + modelRegistry, + messageProvenance, + message, + 'memory' + )) + ) { refuseResolvedSecretProjection({ site: 'memory.messageProvenanceImport', message: MEMORY_CONTENT_REFUSAL, diff --git a/apps/sim/executor/utils/resolved-secret-content-projection.test.ts b/apps/sim/executor/utils/resolved-secret-content-projection.test.ts index 23f1731b0c9..88a38cad883 100644 --- a/apps/sim/executor/utils/resolved-secret-content-projection.test.ts +++ b/apps/sim/executor/utils/resolved-secret-content-projection.test.ts @@ -3,7 +3,9 @@ */ import { describe, expect, it, vi } from 'vitest' import { + createResolvedSecretMatcher, isResolvedSecretModelContentUnchanged, + projectResolvedSecretContent, projectResolvedSecretDiagnosticError, projectResolvedSecretModelContent, projectResolvedSecretModelJsonContent, @@ -175,7 +177,7 @@ describe('projectResolvedSecretModelContent', () => { }) }) - it('projects exact typed primitive secrets without rewriting unrelated primitives', () => { + it('projects exact typed numeric secrets, leaving booleans and null identifying nothing', () => { const registry = new ResolvedSecretTraceRegistry([ { name: 'NUMBER', plaintext: '123', encryptedValue: 'number-ciphertext' }, { name: 'BOOLEAN', plaintext: 'true', encryptedValue: 'boolean-ciphertext' }, @@ -200,17 +202,17 @@ describe('projectResolvedSecretModelContent', () => { ).toEqual({ safe: true, value: { - strings: ['{{NUMBER}}', '{{BOOLEAN}}', '{{NULL}}'], + strings: ['{{NUMBER}}', 'true', 'null'], number: '{{NUMBER}}', - boolean: '{{BOOLEAN}}', - nothing: '{{NULL}}', + boolean: true, + nothing: null, unrelatedNumber: 1234, unrelatedBoolean: false, }, }) }) - it.each(['123', 'true'])('keeps projected JSON argument strings valid (%s)', (secret) => { + it.each(['123'])('keeps projected JSON argument strings valid (%s)', (secret) => { const registry = new ResolvedSecretTraceRegistry([ { name: 'TOKEN', plaintext: secret, encryptedValue: 'ciphertext' }, ]) @@ -231,6 +233,26 @@ describe('projectResolvedSecretModelContent', () => { }) }) + it('leaves a boolean-valued secret in a JSON argument string untouched', () => { + const registry = new ResolvedSecretTraceRegistry([ + { name: 'TOKEN', plaintext: 'true', encryptedValue: 'ciphertext' }, + ]) + registry.recordResolved('TOKEN', 'true') + + const projection = projectResolvedSecretModelJsonStrings( + [JSON.stringify({ secret: 'true', converted: true, nested: [true] })], + registry + ) + + expect(projection.safe).toBe(true) + if (!projection.safe || !Array.isArray(projection.value)) return + expect(JSON.parse(projection.value[0] as string)).toEqual({ + secret: 'true', + converted: true, + nested: [true], + }) + }) + it('is stable when a secret literal overlaps its own provenance alias', () => { const registry = new ResolvedSecretTraceRegistry([ { name: 'TOKEN', plaintext: 'TOKEN', encryptedValue: 'ciphertext' }, @@ -464,3 +486,46 @@ describe('projectResolvedSecretDiagnosticError', () => { }) }) }) + +describe('literals too small to identify anything', () => { + const matcher = createResolvedSecretMatcher( + [ + { plaintext: 'false', replacement: '{{BANNER_ENABLED}}' }, + { plaintext: 'xoxb-real-secret-value', replacement: '{{SLACK_TOKEN}}' }, + ], + { preserveNamedProvenanceLabels: true, mode: 'render' } + )! + + const project = (value: unknown) => + projectResolvedSecretContent(value, matcher, 1_000_000, { projectPrimitiveLiterals: true }) + + /** A `*_ENABLED` variable holding `false` once rewrote 2,000 boolean cells in one table read. */ + it('leaves a typed boolean cell alone', () => { + expect(project({ had_error: false, ok: true, missing: null })).toEqual({ + safe: true, + value: { had_error: false, ok: true, missing: null }, + }) + }) + + it('leaves a delimited occurrence inside surrounding text alone', () => { + expect(project({ url: 'https://x?fromUser=false&sort=count' })).toEqual({ + safe: true, + value: { url: 'https://x?fromUser=false&sort=count' }, + }) + }) + + it('still substitutes a real secret sharing the same matcher', () => { + expect(project({ token: 'xoxb-real-secret-value', flag: false })).toEqual({ + safe: true, + value: { token: '{{SLACK_TOKEN}}', flag: false }, + }) + }) + + it('builds no matcher at all when every literal is non-identifying', () => { + expect( + createResolvedSecretMatcher([{ plaintext: 'true', replacement: '{{FLAG}}' }], { + mode: 'render', + }) + ).toBeUndefined() + }) +}) diff --git a/apps/sim/executor/utils/resolved-secret-match-policy.test.ts b/apps/sim/executor/utils/resolved-secret-match-policy.test.ts index 864d920e451..bf79df5754a 100644 --- a/apps/sim/executor/utils/resolved-secret-match-policy.test.ts +++ b/apps/sim/executor/utils/resolved-secret-match-policy.test.ts @@ -4,6 +4,7 @@ import { describe, expect, it } from 'vitest' import { getResolvedSecretMatchPolicy, + isNonIdentifyingSecretLiteral, isWordBoundaryMatch, MIN_UNANCHORED_MATCH_LENGTH, } from '@/executor/utils/resolved-secret-match-policy' @@ -89,3 +90,19 @@ describe('isWordBoundaryMatch', () => { expect(isWordBoundaryMatch('abc', 3, 3)).toBe(true) }) }) + +describe('isNonIdentifyingSecretLiteral', () => { + it.each(['true', 'false', 'null'])( + 'excludes %s, whose value space is too small to identify', + (literal) => { + expect(isNonIdentifyingSecretLiteral(literal)).toBe(true) + } + ) + + it.each(['0', '1', 'False', 'TRUE', 'Null', 'nullish', '', 'hunter2', 'sk_live_abc'])( + 'keeps %s protectable', + (literal) => { + expect(isNonIdentifyingSecretLiteral(literal)).toBe(false) + } + ) +}) diff --git a/apps/sim/executor/utils/resolved-secret-match-policy.ts b/apps/sim/executor/utils/resolved-secret-match-policy.ts index 2bd9ffd0b11..4b572662ed5 100644 --- a/apps/sim/executor/utils/resolved-secret-match-policy.ts +++ b/apps/sim/executor/utils/resolved-secret-match-policy.ts @@ -35,6 +35,37 @@ export type ResolvedSecretMatchPolicy = 'anywhere' | 'boundary' */ export const MIN_UNANCHORED_MATCH_LENGTH = 8 +/** + * Literals that may not match at all, at any offset, because they identify nothing. + * + * This is cardinality, not entropy — the distinction the floor above turns on. An all-`f` HMAC key + * is low-entropy but drawn from an enormous space, so a hit on it is evidence. `false` is drawn + * from a space of two: a hit on it is evidence of nothing, and substituting it protects nothing an + * attacker could not guess by flipping a coin. Meanwhile it rewrites every boolean any workflow + * ever wrote — one deployment turned 2,000 `had_error` cells into `[REDACTED_SECRET]` because a + * `*_BANNER_ENABLED` variable happened to hold `false`. + * + * Exactly the three JSON renderings of a non-string primitive, and nothing else. `0` and `1` are + * deliberately absent: a short numeric secret is entirely plausible where a boolean one is not. + * Matching is case-sensitive because the set is defined by what `String(value)` produces for a + * typed primitive, not by what looks boolean — an environment variable literally holding `False` + * keeps its protection. + * + * The residual is one bit: a variable whose whole value is the string `false` is no longer hidden. + */ +const NON_IDENTIFYING_SECRET_LITERALS: ReadonlySet = new Set(['true', 'false', 'null']) + +/** + * True when a literal carries too little information to be worth protecting anywhere. + * + * Applied where literals are turned into matchers, so it governs detection and substitution alike: + * such a value is never rewritten out of content, and never recorded into durable provenance as + * something a later read must redact. + */ +export function isNonIdentifyingSecretLiteral(plaintext: string): boolean { + return NON_IDENTIFYING_SECRET_LITERALS.has(plaintext) +} + /** * Combining marks count so a substitution cannot split a grapheme cluster. `_` deliberately does * NOT: `sk_live_...` and `user_483920_profile` are the dominant way a secret gets joined into an diff --git a/apps/sim/executor/utils/resolved-secret-matcher.ts b/apps/sim/executor/utils/resolved-secret-matcher.ts index b20aad11653..60080f3249e 100644 --- a/apps/sim/executor/utils/resolved-secret-matcher.ts +++ b/apps/sim/executor/utils/resolved-secret-matcher.ts @@ -1,6 +1,7 @@ import { MAX_INLINE_MATERIALIZATION_BYTES } from '@/lib/execution/payloads/limits' import { getResolvedSecretMatchPolicy, + isNonIdentifyingSecretLiteral, type ResolvedSecretMatchPolicy, satisfiesResolvedSecretMatchPolicy, } from '@/executor/utils/resolved-secret-match-policy' @@ -457,7 +458,12 @@ export function createResolvedSecretMatcher( const replacementByPlaintext = new Map() for (const match of matches) { - if (!match.plaintext) continue + /** + * Dropped before any construction-time check runs, so no later stage can be talked into + * treating one of these as protectable — including the wide-match-set checks below, which + * deliberately ignore the narrow policy. + */ + if (!match.plaintext || isNonIdentifyingSecretLiteral(match.plaintext)) continue const current = replacementByPlaintext.get(match.plaintext) if (current === undefined || compareStrings(match.replacement, current) < 0) { replacementByPlaintext.set(match.plaintext, match.replacement) diff --git a/apps/sim/executor/utils/resolved-secret-trace-registry.test.ts b/apps/sim/executor/utils/resolved-secret-trace-registry.test.ts index 0db5b2e881f..6c2f057e2fb 100644 --- a/apps/sim/executor/utils/resolved-secret-trace-registry.test.ts +++ b/apps/sim/executor/utils/resolved-secret-trace-registry.test.ts @@ -1010,11 +1010,11 @@ describe('ResolvedSecretTraceRegistry', () => { it('conservatively retains every active secret that shares a raw plaintext literal', () => { const registry = new ResolvedSecretTraceRegistry([ - { name: 'FIRST', plaintext: 'true', encryptedValue: 'first-ciphertext' }, - { name: 'SECOND', plaintext: 'true', encryptedValue: 'second-ciphertext' }, + { name: 'FIRST', plaintext: '4815162342', encryptedValue: 'first-ciphertext' }, + { name: 'SECOND', plaintext: '4815162342', encryptedValue: 'second-ciphertext' }, ]) - registry.recordResolved('FIRST', 'true') - registry.recordResolved('SECOND', 'true') + registry.recordResolved('FIRST', '4815162342') + registry.recordResolved('SECOND', '4815162342') const expected = { version: 1 as const, @@ -1024,11 +1024,11 @@ describe('ResolvedSecretTraceRegistry', () => { { name: 'SECOND', encryptedValue: 'second-ciphertext' }, ], } - expect(registry.exportCommittedProvenanceForValue('true')).toEqual(expected) - expect(registry.exportCommittedProvenanceForValue(true)).toEqual(expected) + expect(registry.exportCommittedProvenanceForValue('4815162342')).toEqual(expected) + expect(registry.exportCommittedProvenanceForValue(4815162342)).toEqual(expected) }) - it('exports active numeric, boolean, and null literals crossing a value boundary', () => { + it('exports active numeric literals crossing a value boundary, but not boolean or null', () => { const registry = new ResolvedSecretTraceRegistry([ { name: 'NUMBER', plaintext: '1234', encryptedValue: 'number-ciphertext' }, { name: 'BOOLEAN', plaintext: 'false', encryptedValue: 'boolean-ciphertext' }, @@ -1048,19 +1048,17 @@ describe('ResolvedSecretTraceRegistry', () => { ).toEqual({ version: 1, complete: true, - entries: [ - { encryptedValue: 'boolean-ciphertext' }, - { encryptedValue: 'null-ciphertext' }, - { encryptedValue: 'number-ciphertext' }, - ], + entries: [{ encryptedValue: 'number-ciphertext' }], }) }) - it('marks a bounded cross-boundary scan incomplete when an enumerable accessor is opaque', () => { + it('keeps every candidate when a bounded cross-boundary scan hits an opaque accessor', () => { const registry = new ResolvedSecretTraceRegistry([ { name: 'TOKEN', plaintext: 'secret', encryptedValue: 'ciphertext' }, + { name: 'ABSENT', plaintext: 'never-present', encryptedValue: 'absent-ciphertext' }, ]) registry.recordResolved('TOKEN', 'secret') + registry.recordResolved('ABSENT', 'never-present') const value = {} Object.defineProperty(value, 'opaque', { enumerable: true, @@ -1069,16 +1067,18 @@ describe('ResolvedSecretTraceRegistry', () => { expect(registry.exportProvenanceForValue(value, { anonymous: true })).toEqual({ version: 1, - complete: false, - entries: [], + complete: true, + entries: [{ encryptedValue: 'absent-ciphertext' }, { encryptedValue: 'ciphertext' }], }) }) - it('does not claim a complete cross-boundary scan for opaque large-value refs', () => { + it('keeps every candidate rather than voiding provenance for an opaque large-value ref', () => { const registry = new ResolvedSecretTraceRegistry([ { name: 'TOKEN', plaintext: 'secret', encryptedValue: 'ciphertext' }, + { name: 'ABSENT', plaintext: 'never-present', encryptedValue: 'absent-ciphertext' }, ]) registry.recordResolved('TOKEN', 'secret') + registry.recordResolved('ABSENT', 'never-present') expect( registry.exportProvenanceForValue( @@ -1091,6 +1091,70 @@ describe('ResolvedSecretTraceRegistry', () => { }, { anonymous: true } ) + ).toEqual({ + version: 1, + complete: true, + entries: [{ encryptedValue: 'absent-ciphertext' }, { encryptedValue: 'ciphertext' }], + }) + }) + + it('lets a model input path survive an upstream output the scan could not read', async () => { + const scope = { userId: 'user-1', workspaceId: 'workspace-1' } + const catalog = [ + { name: 'TOKEN', plaintext: 'decrypted:ciphertext', encryptedValue: 'ciphertext' }, + ] + const producer = new ResolvedSecretTraceRegistry(catalog, scope) + producer.recordResolved('TOKEN', 'decrypted:ciphertext') + + /** A block output past the traversal bound, exactly as compaction leaves a large table read. */ + const upstreamOutput = { + rows: Array.from({ length: 5_000 }, (_, index) => ({ + id: `row_${index}`, + a: 'a', + b: 'b', + c: 'c', + d: 'd', + e: 'e', + f: 'f', + g: 'g', + h: 'h', + i: 'i', + j: 'j', + })), + } + const upstreamProvenance = producer.exportCommittedProvenanceForValue(upstreamOutput) + expect(upstreamProvenance.complete).toBe(true) + + const consumer = new ResolvedSecretTraceRegistry(catalog, scope) + await consumer.importProvenanceForValueAtInputPath( + upstreamProvenance, + upstreamOutput, + ['userPrompt'], + { trusted: true } + ) + + const modelFork = consumer.forkForInputPaths([['userPrompt'], ['systemPrompt']]) + expect(modelFork.projectResolvedInputSelection({ userPrompt: 'classify these rows' })).toEqual({ + complete: true, + value: { userPrompt: 'classify these rows' }, + }) + }) + + it('still voids provenance for an unscannable value when the registry cannot vouch', () => { + const registry = new ResolvedSecretTraceRegistry([ + { name: 'TOKEN', plaintext: 'secret', encryptedValue: 'ciphertext' }, + ]) + registry.recordResolved('TOKEN', 'secret') + registry.markIncomplete('unverified-resolved-entry') + + expect( + registry.exportCommittedProvenanceForValue({ + __simLargeValueRef: true, + version: 1, + id: 'lv_ABCDEFGHIJKL', + kind: 'object', + size: 1024, + }) ).toEqual({ version: 1, complete: false, entries: [] }) }) @@ -1110,8 +1174,8 @@ describe('ResolvedSecretTraceRegistry', () => { expect(provenance).toEqual({ version: 1, - complete: false, - entries: [], + complete: true, + entries: [{ encryptedValue: 'ciphertext' }], }) expect(descriptorSnapshotCalls).toBe(0) }) @@ -1508,3 +1572,53 @@ describe('incompleteness diagnostics', () => { expect(logged).not.toContain('MISSING') }) }) + +describe('non-identifying literals in durable provenance', () => { + /** + * The amplifier behind the boolean redaction: once recorded on a row, every later read of that + * table reactivated the value and rewrote every boolean in it. + */ + it('never records a value too small to identify anything', () => { + const registry = new ResolvedSecretTraceRegistry([ + { name: 'BANNER_ENABLED', plaintext: 'false', encryptedValue: 'flag-ciphertext' }, + { name: 'TOKEN', plaintext: 'xoxb-real-secret-value', encryptedValue: 'token-ciphertext' }, + ]) + registry.recordResolved('BANNER_ENABLED', 'false') + registry.recordResolved('TOKEN', 'xoxb-real-secret-value') + + expect(registry.exportProvenanceForValue({ had_error: false, note: 'fromUser=false' })).toEqual( + { version: 1, complete: true, entries: [] } + ) + expect(registry.exportProvenanceForValue({ token: 'xoxb-real-secret-value' })).toEqual({ + version: 1, + complete: true, + entries: [{ name: 'TOKEN', encryptedValue: 'token-ciphertext' }], + }) + }) + + it('still recognizes the internal alias, which names the variable its value cannot', () => { + const registry = new ResolvedSecretTraceRegistry([ + { name: 'BANNER_ENABLED', plaintext: 'false', encryptedValue: 'flag-ciphertext' }, + ]) + registry.recordResolved('BANNER_ENABLED', 'false') + + expect(registry.exportProvenanceForValue({ code: '__var_BANNER_ENABLED' })).toEqual({ + version: 1, + complete: true, + entries: [{ name: 'BANNER_ENABLED', encryptedValue: 'flag-ciphertext' }], + }) + }) + + it('keeps it out of the model matcher so nothing downstream can substitute it', () => { + const registry = new ResolvedSecretTraceRegistry([ + { name: 'BANNER_ENABLED', plaintext: 'false', encryptedValue: 'flag-ciphertext' }, + ]) + registry.recordResolved('BANNER_ENABLED', 'false') + + const snapshot = registry.getModelEgressSnapshot() + expect(snapshot.complete).toBe(true) + if (snapshot.complete) { + expect(snapshot.matches.map((match) => match.plaintext)).not.toContain('false') + } + }) +}) diff --git a/apps/sim/executor/utils/resolved-secret-trace-registry.ts b/apps/sim/executor/utils/resolved-secret-trace-registry.ts index 853573eae6d..377c89d72da 100644 --- a/apps/sim/executor/utils/resolved-secret-trace-registry.ts +++ b/apps/sim/executor/utils/resolved-secret-trace-registry.ts @@ -4,6 +4,7 @@ import { decryptSecret } from '@/lib/core/security/encryption' import { isLargeArrayManifest } from '@/lib/execution/payloads/large-array-manifest-metadata' import { isLargeValueRef } from '@/lib/execution/payloads/large-value-ref' import { MAX_INLINE_MATERIALIZATION_BYTES } from '@/lib/execution/payloads/limits' +import { isNonIdentifyingSecretLiteral } from '@/executor/utils/resolved-secret-match-policy' import { createResolvedSecretMatcher, OPAQUE_RESOLVED_SECRET_REPLACEMENT, @@ -38,6 +39,8 @@ export type ResolvedSecretIncompletenessReason = | 'value-provenance-untrusted' | 'value-provenance-import-failed' | 'value-provenance-filter-incomplete' + | 'durable-provenance-unknown' + | 'durable-provenance-malformed' | 'unspecified' /** @@ -64,6 +67,7 @@ const ORIGINATING_FAULT_REASONS = new Set([ 'tool-call-scope-mismatch', 'value-provenance-untrusted', 'value-provenance-import-failed', + 'durable-provenance-malformed', ]) /** @@ -161,13 +165,17 @@ interface ResolvedInputPathState { interface PreparedProvenanceFilter { candidatesByScanLiteral: ReadonlyMap candidatesByAlias: ReadonlyMap - candidateEntryKeys: ReadonlySet + candidateEntries: ReadonlyMap matcher?: ResolvedSecretMatcher } +/** + * Carries the candidate entries on both arms, because they are the answer whenever narrowing is + * unavailable — including when the matcher itself could not be built. + */ type PreparedProvenanceFilterResult = | { complete: true; filter: PreparedProvenanceFilter } - | { complete: false } + | { complete: false; candidateEntries: ReadonlyMap } export interface ImportResolvedSecretTraceProvenanceOptions { trusted: boolean @@ -1296,7 +1304,12 @@ export class ResolvedSecretTraceRegistry { private buildMatches(entries: Iterable): readonly ResolvedSecretTraceMatch[] { const candidatesByPlaintext = new Map() for (const entry of entries) { - if (entry.plaintext.length === 0) continue + /** + * Dropped here too, not only inside the matcher, so a literal that will never be substituted + * also never counts toward the matcher capacity bound or appears to a snapshot reader as + * something this registry protects. + */ + if (entry.plaintext.length === 0 || isNonIdentifyingSecretLiteral(entry.plaintext)) continue const candidates = candidatesByPlaintext.get(entry.plaintext) ?? [] candidates.push(entry) candidatesByPlaintext.set(entry.plaintext, candidates) @@ -1495,16 +1508,21 @@ export class ResolvedSecretTraceRegistry { } private prepareProvenanceFilter( - candidateEntries: Iterable + sourceEntries: Iterable ): PreparedProvenanceFilterResult { const candidatesByPlaintext = new Map() - const sortedCandidateEntries = [...candidateEntries].sort( + const sortedCandidateEntries = [...sourceEntries].sort( (left, right) => compareStrings(left.name, right.name) || compareStrings(left.encryptedValue, right.encryptedValue) ) for (const entry of sortedCandidateEntries) { - if (entry.plaintext.length === 0) continue + /** + * Excluded from scan literals as well as from the matcher, so such a value is never recorded + * into durable provenance as something a later read must redact. A named entry still joins + * the alias loop below — `__var_NAME` identifies the variable even when its value does not. + */ + if (entry.plaintext.length === 0 || isNonIdentifyingSecretLiteral(entry.plaintext)) continue const candidates = candidatesByPlaintext.get(entry.plaintext) ?? [] const entryKey = activeEntryKey(entry) if (!candidates.some((candidate) => activeEntryKey(candidate) === entryKey)) { @@ -1514,7 +1532,7 @@ export class ResolvedSecretTraceRegistry { } const candidatesByScanLiteral = new Map() - const candidateEntryKeys = new Set() + const candidateEntries = new Map() const addScanLiteral = (literal: string, entry: ActiveSecretEntry): void => { if (literal.length === 0) return const candidates = candidatesByScanLiteral.get(literal) ?? [] @@ -1523,7 +1541,7 @@ export class ResolvedSecretTraceRegistry { candidates.push(entry) candidatesByScanLiteral.set(literal, candidates) } - candidateEntryKeys.add(entryKey) + candidateEntries.set(entryKey, entry) } for (const candidates of candidatesByPlaintext.values()) { for (const entry of candidates) { @@ -1542,7 +1560,7 @@ export class ResolvedSecretTraceRegistry { candidates.push(entry) candidatesByAlias.set(alias, candidates) } - candidateEntryKeys.add(entryKey) + candidateEntries.set(entryKey, entry) } let matcher: ResolvedSecretMatcher | undefined @@ -1555,7 +1573,7 @@ export class ResolvedSecretTraceRegistry { error: getErrorMessage(error, 'Unknown error'), candidateCount: candidatesByScanLiteral.size, }) - return { complete: false } + return { complete: false, candidateEntries } } return { @@ -1563,12 +1581,49 @@ export class ResolvedSecretTraceRegistry { filter: { candidatesByScanLiteral, candidatesByAlias, - candidateEntryKeys, + candidateEntries, ...(matcher ? { matcher } : {}), }, } } + /** Builds the envelope for one selected entry set; only this registry's own state can void it. */ + private provenanceForSelectedEntries( + entries: ReadonlyMap, + options: ExportResolvedSecretTraceProvenanceForValueOptions + ): ResolvedSecretTraceProvenanceV1 { + const complete = !this.isPermanentlyIncomplete() + return { + version: 1, + complete, + entries: complete + ? this.buildProvenanceEntries([...entries.values()], options.anonymous) + : [], + ...(this.scope ? { scope: cloneProvenanceScope(this.scope) } : {}), + } + } + + /** + * Answers a value the bounded scan could not read in full by keeping every candidate entry. + * + * Narrowing exists to stop content that provably carries no secret from being over-redacted; it + * is not what makes an envelope trustworthy. The candidates are already the trusted answer to + * "which secrets could this value carry", so an unreadable value — an offloaded large-value ref + * the scan cannot see through, a payload past the traversal bound, a hostile accessor — degrades + * to no narrowing rather than to unknown provenance. + * + * Reporting unknown here is what let a size threshold behave like a permanent fault: the flag + * travels onto the producing block's state, and every model boundary that later consumes that + * output refuses, with nothing telling the author the cause was payload volume rather than a + * secret. Over-approximating costs extra redaction; it can never under-redact. + */ + private unnarrowedProvenance( + candidateEntries: ReadonlyMap, + options: ExportResolvedSecretTraceProvenanceForValueOptions + ): ResolvedSecretTraceProvenanceV1 { + return this.provenanceForSelectedEntries(candidateEntries, options) + } + private exportProvenanceForValueWithPreparedFilter( value: unknown, prepared: PreparedProvenanceFilterResult, @@ -1583,37 +1638,22 @@ export class ResolvedSecretTraceRegistry { options: ExportResolvedSecretTraceProvenanceForValueOptions ): ResolvedSecretTraceProvenanceV1 { if (!prepared.complete) { - return { - version: 1, - complete: false, - entries: [], - ...(this.scope ? { scope: cloneProvenanceScope(this.scope) } : {}), - } + return this.unnarrowedProvenance(prepared.candidateEntries, options) } - const { candidatesByScanLiteral, candidatesByAlias, candidateEntryKeys, matcher } = + const { candidatesByScanLiteral, candidatesByAlias, candidateEntries, matcher } = prepared.filter const matchedEntries = new Map() const pendingValues: unknown[] = [] try { for (const value of values) { if (pendingValues.length >= MAX_PROVENANCE_FILTER_NODES) { - return { - version: 1, - complete: false, - entries: [], - ...(this.scope ? { scope: cloneProvenanceScope(this.scope) } : {}), - } + return this.unnarrowedProvenance(candidateEntries, options) } pendingValues.push(value) } } catch { - return { - version: 1, - complete: false, - entries: [], - ...(this.scope ? { scope: cloneProvenanceScope(this.scope) } : {}), - } + return this.unnarrowedProvenance(candidateEntries, options) } const visited = new WeakSet() let scannedNodes = 0 @@ -1661,7 +1701,7 @@ export class ResolvedSecretTraceRegistry { if (scannedNodes + pendingValues.length >= MAX_PROVENANCE_FILTER_NODES) return false scannedNodes++ if (!scanString(key)) return false - if (matchedEntries.size >= candidateEntryKeys.size) return true + if (matchedEntries.size >= candidateEntries.size) return true if ('value' in descriptor) { if (scannedNodes + pendingValues.length >= MAX_PROVENANCE_FILTER_NODES) return false @@ -1672,7 +1712,7 @@ export class ResolvedSecretTraceRegistry { return true } - while (pendingValues.length > 0 && matchedEntries.size < candidateEntryKeys.size) { + while (pendingValues.length > 0 && matchedEntries.size < candidateEntries.size) { const current = pendingValues.pop() scannedNodes++ if (scannedNodes > MAX_PROVENANCE_FILTER_NODES) { @@ -1708,9 +1748,9 @@ export class ResolvedSecretTraceRegistry { scanComplete = false break } - if (matchedEntries.size >= candidateEntryKeys.size) break + if (matchedEntries.size >= candidateEntries.size) break } - if (!scanComplete || matchedEntries.size >= candidateEntryKeys.size) break + if (!scanComplete || matchedEntries.size >= candidateEntries.size) break for (const key in current as Record) { enumeratedProperties++ @@ -1725,7 +1765,7 @@ export class ResolvedSecretTraceRegistry { scanComplete = false break } - if (matchedEntries.size >= candidateEntryKeys.size) break + if (matchedEntries.size >= candidateEntries.size) break } if (!scanComplete) break } catch { @@ -1734,16 +1774,9 @@ export class ResolvedSecretTraceRegistry { } } - const complete = !this.isPermanentlyIncomplete() && scanComplete - const entries = complete - ? this.buildProvenanceEntries([...matchedEntries.values()], options.anonymous) - : [] - return { - version: 1, - complete, - entries, - ...(this.scope ? { scope: cloneProvenanceScope(this.scope) } : {}), - } + return scanComplete + ? this.provenanceForSelectedEntries(matchedEntries, options) + : this.unnarrowedProvenance(candidateEntries, options) } private collectInputPathEntryKeys(paths: readonly ResolvedSecretInputPath[]): Set { diff --git a/apps/sim/lib/copilot/request/tools/resolved-secret-result.test.ts b/apps/sim/lib/copilot/request/tools/resolved-secret-result.test.ts index e138881aa12..2cb2066c699 100644 --- a/apps/sim/lib/copilot/request/tools/resolved-secret-result.test.ts +++ b/apps/sim/lib/copilot/request/tools/resolved-secret-result.test.ts @@ -227,7 +227,7 @@ describe('projectToolResultForCopilot', () => { ).toEqual({ success: true, output: { result: encoded } }) }) - it('projects exact typed primitive secrets and the same values as strings', () => { + it('projects exact typed numeric secrets, leaving booleans and null identifying nothing', () => { const registry = new ResolvedSecretTraceRegistry([ { name: 'NUMBER', plaintext: '123', encryptedValue: 'number-ciphertext' }, { name: 'BOOLEAN', plaintext: 'true', encryptedValue: 'boolean-ciphertext' }, @@ -256,11 +256,11 @@ describe('projectToolResultForCopilot', () => { success: true, output: { number: '{{NUMBER}}', - boolean: '{{BOOLEAN}}', - nothing: '{{NULL}}', + boolean: true, + nothing: null, numberText: '{{NUMBER}}', - booleanText: '{{BOOLEAN}}', - nilText: '{{NULL}}', + booleanText: 'true', + nilText: 'null', }, }) }) diff --git a/apps/sim/lib/copilot/request/tools/tables.test.ts b/apps/sim/lib/copilot/request/tools/tables.test.ts index f75dbdde3f6..cc9faafbff9 100644 --- a/apps/sim/lib/copilot/request/tools/tables.test.ts +++ b/apps/sim/lib/copilot/request/tools/tables.test.ts @@ -569,7 +569,7 @@ describe('maybeWriteReadCsvToTable', () => { col_status: { version: 1, complete: true, - entries: [{ name: 'BOOLEAN', encryptedValue: 'encrypted-boolean' }], + entries: [], scope: { userId: 'user-1', workspaceId: 'workspace-1' }, }, }, @@ -634,7 +634,7 @@ describe('maybeWriteReadCsvToTable', () => { ) }) - it('preserves numeric and boolean cells while recording their provenance', async () => { + it('preserves numeric and boolean cells, recording provenance only for the identifying one', async () => { const registry = new ResolvedSecretTraceRegistry( [ { name: 'NUMBER', plaintext: '123', encryptedValue: 'encrypted-number' }, @@ -666,9 +666,7 @@ describe('maybeWriteReadCsvToTable', () => { col_age: expect.objectContaining({ entries: [{ name: 'NUMBER', encryptedValue: 'encrypted-number' }], }), - col_active: expect.objectContaining({ - entries: [{ name: 'BOOLEAN', encryptedValue: 'encrypted-boolean' }], - }), + col_active: expect.objectContaining({ entries: [] }), }), }), ], diff --git a/apps/sim/lib/core/config/env.ts b/apps/sim/lib/core/config/env.ts index e0fcf321283..c8f365e8eb9 100644 --- a/apps/sim/lib/core/config/env.ts +++ b/apps/sim/lib/core/config/env.ts @@ -109,6 +109,7 @@ export const env = createEnv({ PII_REDACTION: z.boolean().optional(), // Redact PII from workflow logs via configurable Data Retention rules (Presidio at the logger persist choke point) and expose the Data Retention config UI PII_GRANULAR_REDACTION: z.boolean().optional(), // Expose the execution-altering PII redaction stages (redact workflow input + block outputs in-flight) in the Data Retention config; layered on top of PII_REDACTION TRIGGER_EU_REGION: z.boolean().optional(), // Route Trigger.dev runs to eu-central-1 instead of the default us-east-1 (fallback for the trigger-eu-region flag when AppConfig is not the source of truth) + DURABLE_SECRET_PROVENANCE_ENFORCED_SURFACES: z.string().optional(), // Durable surfaces where unrecorded secret provenance fails the run instead of logging a warning: "all", or a comma-separated subset of memory,table-row,knowledge,workspace-file (default: none enforced) // Table feature limits (per plan). Apply when billing is disabled (free tier defaults) or for billed plans. FREE_TABLES_LIMIT: z.number().optional(), // Max user tables per workspace on free tier (default: 5) diff --git a/apps/sim/lib/execution/durable-secret-provenance-enforcement.test.ts b/apps/sim/lib/execution/durable-secret-provenance-enforcement.test.ts new file mode 100644 index 00000000000..46310bb9b29 --- /dev/null +++ b/apps/sim/lib/execution/durable-secret-provenance-enforcement.test.ts @@ -0,0 +1,92 @@ +/** + * @vitest-environment node + */ +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const { mockEnv, mockLogger } = vi.hoisted(() => ({ + mockEnv: {} as Record, + mockLogger: { info: vi.fn(), warn: vi.fn(), error: vi.fn(), debug: vi.fn() }, +})) + +vi.mock('@/lib/core/config/env', () => ({ env: mockEnv })) +vi.mock('@sim/logger', () => ({ createLogger: () => mockLogger })) + +import { + DURABLE_SECRET_PROVENANCE_SURFACES, + isDurableSecretProvenanceEnforced, + reportUnrecordedDurableProvenance, + resetDurableSecretProvenanceEnforcementCache, +} from '@/lib/execution/durable-secret-provenance-enforcement' + +function configure(value: string | undefined): void { + mockEnv.DURABLE_SECRET_PROVENANCE_ENFORCED_SURFACES = value + resetDurableSecretProvenanceEnforcementCache() +} + +describe('durable secret provenance enforcement', () => { + beforeEach(() => { + vi.clearAllMocks() + configure(undefined) + }) + + it('enforces nothing by default, so unrecorded provenance warns instead of latching', () => { + for (const surface of DURABLE_SECRET_PROVENANCE_SURFACES) { + expect(isDurableSecretProvenanceEnforced(surface)).toBe(false) + } + }) + + it('closes one surface at a time without touching the others', () => { + configure('table-row') + + expect(isDurableSecretProvenanceEnforced('table-row')).toBe(true) + expect(isDurableSecretProvenanceEnforced('memory')).toBe(false) + expect(isDurableSecretProvenanceEnforced('knowledge')).toBe(false) + }) + + it('accepts a comma-separated subset, ignoring case and padding', () => { + configure(' Memory , TABLE-ROW ') + + expect(isDurableSecretProvenanceEnforced('memory')).toBe(true) + expect(isDurableSecretProvenanceEnforced('table-row')).toBe(true) + expect(isDurableSecretProvenanceEnforced('knowledge')).toBe(false) + }) + + it('closes every surface on "all"', () => { + configure('all') + + for (const surface of DURABLE_SECRET_PROVENANCE_SURFACES) { + expect(isDurableSecretProvenanceEnforced(surface)).toBe(true) + } + }) + + it('reports an unrecognized surface rather than silently enforcing nothing', () => { + configure('memory,workspace-file') + + expect(isDurableSecretProvenanceEnforced('memory')).toBe(true) + expect(mockLogger.error).toHaveBeenCalledWith( + 'Ignoring unrecognized durable secret provenance surfaces', + expect.objectContaining({ unrecognized: ['workspace-file'] }) + ) + }) + + it('reports at error with the surface, cause, and affected count so it survives every LOG_LEVEL default', () => { + reportUnrecordedDurableProvenance({ + surface: 'table-row', + cause: 'row-sidecar-not-exact', + affectedCount: 8, + workspaceId: 'workspace-1', + }) + + expect(mockLogger.warn).not.toHaveBeenCalled() + expect(mockLogger.error).toHaveBeenCalledWith( + 'Proceeding on unrecorded durable secret provenance', + { + surface: 'table-row', + cause: 'row-sidecar-not-exact', + enforced: false, + affectedCount: 8, + workspaceId: 'workspace-1', + } + ) + }) +}) diff --git a/apps/sim/lib/execution/durable-secret-provenance-enforcement.ts b/apps/sim/lib/execution/durable-secret-provenance-enforcement.ts new file mode 100644 index 00000000000..8489ab66424 --- /dev/null +++ b/apps/sim/lib/execution/durable-secret-provenance-enforcement.ts @@ -0,0 +1,106 @@ +import { createLogger } from '@sim/logger' +import { env } from '@/lib/core/config/env' + +const logger = createLogger('DurableSecretProvenanceEnforcement') + +/** + * Durable stores that can hand a run a value whose secret provenance was never recorded. + * + * Named by the call site rather than derived, so the surface survives refactors and stays + * greppable — the same convention the projection-refusal `site` strings use. + */ +export const DURABLE_SECRET_PROVENANCE_SURFACES = ['memory', 'table-row', 'knowledge'] as const + +export type DurableSecretProvenanceSurface = (typeof DURABLE_SECRET_PROVENANCE_SURFACES)[number] + +/** Reads the configured surfaces once; an unrecognized name is reported rather than assumed. */ +function resolveEnforcedSurfaces(): ReadonlySet { + const configured = env.DURABLE_SECRET_PROVENANCE_ENFORCED_SURFACES?.trim() + if (!configured) return new Set() + + const requested = configured + .split(',') + .map((entry) => entry.trim().toLowerCase()) + .filter((entry) => entry.length > 0) + if (requested.includes('all')) return new Set(DURABLE_SECRET_PROVENANCE_SURFACES) + + const enforced = new Set() + const unrecognized: string[] = [] + for (const entry of requested) { + const surface = DURABLE_SECRET_PROVENANCE_SURFACES.find((candidate) => candidate === entry) + if (surface) enforced.add(surface) + else unrecognized.push(entry) + } + if (unrecognized.length > 0) { + logger.error('Ignoring unrecognized durable secret provenance surfaces', { + unrecognized, + supported: [...DURABLE_SECRET_PROVENANCE_SURFACES], + }) + } + return enforced +} + +let enforcedSurfaces: ReadonlySet | undefined + +/** + * True when unrecorded provenance from this surface must fail the run rather than warn. + * + * Nothing is enforced by default. `unknown` provenance means "nobody recorded what secrets this + * value carries", which is the same thing a pre-tracking legacy row says — and legacy rows are read + * as carrying none. Enforcing one and not the other made an *aware* writer that momentarily could + * not vouch strictly worse than an unaware one: the row it wrote latched every run that later read + * it, and each latched run wrote more such rows, so a workspace could not recover without a data + * repair. + * + * Warning instead keeps that state visible and measurable while the writers that produce it are + * fixed. The sidecar still records `unknown` faithfully, so a surface can be closed back up once + * its writers stop losing provenance — and the rows that will start failing are countable from the + * sidecar table before the switch is thrown. This is a deliberate posture: an unenforced surface + * can under-redact a value whose provenance was lost. + * + * Set `DURABLE_SECRET_PROVENANCE_ENFORCED_SURFACES` to `all`, or to a comma-separated subset of + * {@link DURABLE_SECRET_PROVENANCE_SURFACES}, to close a surface. + * + * Workspace files are deliberately not a surface here: their unknown check sits in the callers + * rather than in the shared import, and one unknown file locks vfs reads, chat attachments, sandbox + * mounts, and the file tool routes at once. They stay fail-closed until that is its own decision. + */ +export function isDurableSecretProvenanceEnforced( + surface: DurableSecretProvenanceSurface +): boolean { + enforcedSurfaces ??= resolveEnforcedSurfaces() + return enforcedSurfaces.has(surface) +} + +export interface UnrecordedDurableProvenanceReport { + surface: DurableSecretProvenanceSurface + /** What the surface could not vouch for, e.g. `sidecar-status-unknown`. Always a static literal. */ + cause: string + /** How many records in this one read were unrecorded, when the caller reads a page at a time. */ + affectedCount?: number + workspaceId?: string +} + +/** + * Records that a read proceeded on provenance nobody wrote down. + * + * Error, not warn, for the same reason the originating-fault reasons use it: error is the only + * level that survives every default the logger falls back to — production, test, and a self-hosted + * chart that sets no `LOG_LEVEL`. A surface stays open on the strength of this line being visible + * and trending to zero, so a level that a deployment can silently filter would leave the posture + * unmeasured. It is deliberately noisy on an affected workspace; that is the signal. + */ +export function reportUnrecordedDurableProvenance(report: UnrecordedDurableProvenanceReport): void { + logger.error('Proceeding on unrecorded durable secret provenance', { + surface: report.surface, + cause: report.cause, + enforced: false, + ...(report.affectedCount !== undefined ? { affectedCount: report.affectedCount } : {}), + ...(report.workspaceId ? { workspaceId: report.workspaceId } : {}), + }) +} + +/** Test seam: forces the next read to re-resolve the env-configured surfaces. */ +export function resetDurableSecretProvenanceEnforcementCache(): void { + enforcedSurfaces = undefined +} diff --git a/apps/sim/lib/execution/durable-secret-provenance.test.ts b/apps/sim/lib/execution/durable-secret-provenance.test.ts index 75d75d22d38..00a29654067 100644 --- a/apps/sim/lib/execution/durable-secret-provenance.test.ts +++ b/apps/sim/lib/execution/durable-secret-provenance.test.ts @@ -1,12 +1,26 @@ /** * @vitest-environment node */ -import { describe, expect, it } from 'vitest' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const { mockIsEnforced, mockReport } = vi.hoisted(() => ({ + mockIsEnforced: vi.fn(() => false), + mockReport: vi.fn(), +})) + +vi.mock('@/lib/execution/durable-secret-provenance-enforcement', () => ({ + DURABLE_SECRET_PROVENANCE_SURFACES: ['memory', 'table-row', 'knowledge'], + isDurableSecretProvenanceEnforced: mockIsEnforced, + reportUnrecordedDurableProvenance: mockReport, +})) + import { durableSecretProvenanceFromPrivateBundle, filterDurableSecretProvenanceBySourceValues, hashDurableSecretProvenanceValue, + importDurableSecretProvenance, } from '@/lib/execution/durable-secret-provenance' +import { ResolvedSecretTraceRegistry } from '@/executor/utils/resolved-secret-trace-registry' function privateBundle(scope?: { userId: string; workspaceId?: string }) { return { @@ -133,3 +147,54 @@ describe('private durable provenance scope admission', () => { ).toBeUndefined() }) }) + +describe('importing unrecorded durable provenance', () => { + const UNKNOWN = { status: 'unknown' } as const + + beforeEach(() => { + vi.clearAllMocks() + mockIsEnforced.mockReturnValue(false) + }) + + it('warns and leaves the registry able to vouch when the surface is not enforced', async () => { + const registry = new ResolvedSecretTraceRegistry() + + await expect( + importDurableSecretProvenance(registry, UNKNOWN, undefined, 'memory') + ).resolves.toBe(true) + expect(registry.isPermanentlyIncomplete()).toBe(false) + expect(mockReport).toHaveBeenCalledWith({ + surface: 'memory', + cause: 'durable-provenance-unknown', + }) + }) + + it('latches the registry once that surface is closed', async () => { + mockIsEnforced.mockReturnValue(true) + const registry = new ResolvedSecretTraceRegistry() + + await expect( + importDurableSecretProvenance(registry, UNKNOWN, undefined, 'memory') + ).resolves.toBe(false) + expect(registry.isPermanentlyIncomplete()).toBe(true) + expect(mockReport).not.toHaveBeenCalled() + }) + + it('latches for a caller that has not declared a surface', async () => { + const registry = new ResolvedSecretTraceRegistry() + + await expect(importDurableSecretProvenance(registry, UNKNOWN)).resolves.toBe(false) + expect(registry.isPermanentlyIncomplete()).toBe(true) + }) + + it('never relaxes a malformed sidecar, which is a fault rather than missing data', async () => { + const registry = new ResolvedSecretTraceRegistry() + const malformed = { status: 'exact', entries: [{ encryptedValue: '' }] } as never + + await expect( + importDurableSecretProvenance(registry, malformed, undefined, 'memory') + ).resolves.toBe(false) + expect(registry.isPermanentlyIncomplete()).toBe(true) + expect(mockReport).not.toHaveBeenCalled() + }) +}) diff --git a/apps/sim/lib/execution/durable-secret-provenance.ts b/apps/sim/lib/execution/durable-secret-provenance.ts index 334f4cfa07d..edd4a1b7df8 100644 --- a/apps/sim/lib/execution/durable-secret-provenance.ts +++ b/apps/sim/lib/execution/durable-secret-provenance.ts @@ -1,5 +1,10 @@ import { createHash } from 'node:crypto' import type { DurableSecretProvenanceEntry } from '@sim/db/schema' +import { + type DurableSecretProvenanceSurface, + isDurableSecretProvenanceEnforced, + reportUnrecordedDurableProvenance, +} from '@/lib/execution/durable-secret-provenance-enforcement' import { isPrivateSecretProvenanceBundleV1, type PrivateSecretProvenanceBundleV1, @@ -191,19 +196,34 @@ export function filterDurableSecretProvenanceBySourceValues( return entries ? { status: 'exact', entries } : { status: 'unknown' } } -/** Imports durable entries into a model-bound registry, preserving source-scope anonymity. */ +/** + * Imports durable entries into a model-bound registry, preserving source-scope anonymity. + * + * `surface` selects the enforcement policy for provenance nobody recorded. Omitting it enforces, + * which is the right default for a caller that has not been reviewed against + * {@link isDurableSecretProvenanceEnforced} yet. + */ export async function importDurableSecretProvenance( registry: ResolvedSecretTraceRegistry, provenance: DurableSecretProvenance, - value?: unknown + value?: unknown, + surface?: DurableSecretProvenanceSurface ): Promise { if (provenance.status === 'unknown') { - registry.markIncomplete() + if (surface && !isDurableSecretProvenanceEnforced(surface)) { + reportUnrecordedDurableProvenance({ surface, cause: 'durable-provenance-unknown' }) + return true + } + registry.markIncomplete('durable-provenance-unknown') return false } const entries = normalizeDurableSecretProvenanceEntries(provenance.entries) if (!entries) { - registry.markIncomplete() + /** + * Malformed is not unrecorded: a sidecar that exists but cannot be parsed is a fault, and no + * enforcement policy relaxes it. + */ + registry.markIncomplete('durable-provenance-malformed') return false } diff --git a/apps/sim/lib/knowledge/secret-provenance.ts b/apps/sim/lib/knowledge/secret-provenance.ts index 03802733807..139fb0adc44 100644 --- a/apps/sim/lib/knowledge/secret-provenance.ts +++ b/apps/sim/lib/knowledge/secret-provenance.ts @@ -409,7 +409,7 @@ export async function loadKnowledgeDocumentSecretRegistry( tracked: row.secretProvenanceVersion === 1 || currentSourceFileProvenance !== undefined, } const registry = new ResolvedSecretTraceRegistry([], scope) - if (!(await importDurableSecretProvenance(registry, provenance))) { + if (!(await importDurableSecretProvenance(registry, provenance, undefined, 'knowledge'))) { throw new Error('Knowledge document secret provenance is unavailable') } return { registry, provenance, tracked: true } @@ -503,7 +503,9 @@ export async function importKnowledgePersistedResponseSecretProvenance(options: readBoundKnowledgeDocumentSecretProvenance({ ...row, source }), source ) - if (!(await importDurableSecretProvenance(options.registry, provenance, item.value))) { + if ( + !(await importDurableSecretProvenance(options.registry, provenance, item.value, 'knowledge')) + ) { return false } } @@ -515,7 +517,9 @@ export async function importKnowledgePersistedResponseSecretProvenance(options: return false } const provenance = readBoundKnowledgeEmbeddingSecretProvenance(row) - if (!(await importDurableSecretProvenance(options.registry, provenance, item.value))) { + if ( + !(await importDurableSecretProvenance(options.registry, provenance, item.value, 'knowledge')) + ) { return false } } @@ -567,7 +571,14 @@ export async function importKnowledgeSearchResultSecretProvenance(options: { return { imported: false, documentMetadata: {} } } const provenance = readBoundKnowledgeEmbeddingSecretProvenance(row) - if (!(await importDurableSecretProvenance(options.registry, provenance, result.content))) { + if ( + !(await importDurableSecretProvenance( + options.registry, + provenance, + result.content, + 'knowledge' + )) + ) { return { imported: false, documentMetadata: {} } } } diff --git a/apps/sim/lib/table/rows/secret-provenance.test.ts b/apps/sim/lib/table/rows/secret-provenance.test.ts index 20240b69f2a..c9f0a948a55 100644 --- a/apps/sim/lib/table/rows/secret-provenance.test.ts +++ b/apps/sim/lib/table/rows/secret-provenance.test.ts @@ -5,6 +5,18 @@ import { userTableDefinitions, userTableRows } from '@sim/db/schema' import { dbChainMock, dbChainMockFns, queueTableRows, resetDbChainMock } from '@sim/testing' import { eq } from 'drizzle-orm' import { beforeEach, describe, expect, it, vi } from 'vitest' + +const { mockIsEnforced, mockReport } = vi.hoisted(() => ({ + mockIsEnforced: vi.fn(() => false), + mockReport: vi.fn(), +})) + +vi.mock('@/lib/execution/durable-secret-provenance-enforcement', () => ({ + DURABLE_SECRET_PROVENANCE_SURFACES: ['memory', 'table-row', 'knowledge'], + isDurableSecretProvenanceEnforced: mockIsEnforced, + reportUnrecordedDurableProvenance: mockReport, +})) + import type { DbTransaction } from '@/lib/table/planner' import { classifyTableRowSecretProvenanceForCopy, @@ -46,6 +58,7 @@ describe('table row secret provenance', () => { beforeEach(() => { vi.clearAllMocks() resetDbChainMock() + mockIsEnforced.mockReturnValue(false) }) it('checks a version-pinned table with one bounded unsafe-row query', async () => { @@ -204,7 +217,8 @@ describe('table row secret provenance', () => { }) }) - it('fails closed for stale tracked rows instead of returning partial provenance', async () => { + it('fails closed for stale tracked rows once the table-row surface is enforced', async () => { + mockIsEnforced.mockReturnValue(true) queueTableRows(userTableRows, [ { id: 'tracked-row', @@ -230,6 +244,62 @@ describe('table row secret provenance', () => { }) }) + /** + * The shape that broke production: one unrecorded row in a page voided the whole read, and a + * page is what a `query_rows` block hands downstream, so every later model boundary refused. + */ + it('keeps a page readable when one row is unrecorded, without dropping its siblings', async () => { + queueTableRows(userTableRows, [ + { + id: 'unknown-row', + updatedAt: ROW_UPDATED_AT, + secretProvenanceVersion: 1, + sidecarRowId: 'unknown-row', + sidecarStatus: 'unknown', + sidecarEntries: [], + sidecarIsCurrent: true, + }, + { + id: 'tracked-row', + updatedAt: ROW_UPDATED_AT, + secretProvenanceVersion: 1, + sidecarRowId: 'tracked-row', + sidecarStatus: 'exact', + sidecarEntries: [ + { + columnId: 'secret-column', + encryptedValue: 'encrypted-local', + name: 'LOCAL_SECRET', + sourceUserId: 'user-1', + sourceWorkspaceId: 'workspace-1', + }, + ], + sidecarIsCurrent: true, + }, + ]) + + await expect( + loadTableRowSecretProvenance( + [ + { id: 'unknown-row', updatedAt: ROW_UPDATED_AT }, + { id: 'tracked-row', updatedAt: ROW_UPDATED_AT }, + ], + { userId: 'user-1', workspaceId: 'workspace-1' } + ) + ).resolves.toEqual({ + version: 1, + complete: true, + entries: [{ encryptedValue: 'encrypted-local', name: 'LOCAL_SECRET' }], + scope: { userId: 'user-1', workspaceId: 'workspace-1' }, + }) + expect(mockReport).toHaveBeenCalledWith({ + surface: 'table-row', + cause: 'row-sidecar-not-exact', + affectedCount: 1, + workspaceId: 'workspace-1', + }) + }) + it('rejects contradictory duplicate row crossings before reading provenance', async () => { await expect( loadTableRowSecretProvenance( diff --git a/apps/sim/lib/table/rows/secret-provenance.ts b/apps/sim/lib/table/rows/secret-provenance.ts index 126c3c8ea84..160bffa73c1 100644 --- a/apps/sim/lib/table/rows/secret-provenance.ts +++ b/apps/sim/lib/table/rows/secret-provenance.ts @@ -6,6 +6,10 @@ import { userTableRows, } from '@sim/db/schema' import { and, asc, eq, gt, inArray, type SQL, sql } from 'drizzle-orm' +import { + isDurableSecretProvenanceEnforced, + reportUnrecordedDurableProvenance, +} from '@/lib/execution/durable-secret-provenance-enforcement' import type { DbExecutor, DbTransaction } from '@/lib/table/planner' import type { RowData, TableRowSecretProvenanceWrite } from '@/lib/table/types' import { @@ -765,6 +769,7 @@ export async function loadTableRowSecretProvenance( const currentById = new Map(currentRows.map((row) => [row.id, row])) const storedEntries: StoredTableRowSecretProvenanceEntry[] = [] + let unrecordedRowCount = 0 for (const rowId of rowIds) { const current = currentById.get(rowId) const crossing = crossingById.get(rowId) @@ -777,7 +782,16 @@ export async function loadTableRowSecretProvenance( current.sidecarStatus !== 'exact' || !current.sidecarIsCurrent ) { - return { version: 1, complete: false, entries: [], scope } + /** + * One such row would otherwise void the whole page, and a page is what a `query_rows` block + * hands downstream — so a single row nobody recorded provenance for latched every run that + * read the table. Unenforced, the row contributes nothing, exactly like the legacy row above. + */ + if (isDurableSecretProvenanceEnforced('table-row')) { + return { version: 1, complete: false, entries: [], scope } + } + unrecordedRowCount += 1 + continue } const parsed = normalizeStoredEntries(current.sidecarEntries) if (!parsed) return { version: 1, complete: false, entries: [], scope } @@ -791,6 +805,15 @@ export async function loadTableRowSecretProvenance( } } + if (unrecordedRowCount > 0) { + reportUnrecordedDurableProvenance({ + surface: 'table-row', + cause: 'row-sidecar-not-exact', + affectedCount: unrecordedRowCount, + ...(scope.workspaceId ? { workspaceId: scope.workspaceId } : {}), + }) + } + const entries = aggregateStoredEntries(storedEntries, scope) if (!entries) return { version: 1, complete: false, entries: [], scope } const provenance: ResolvedSecretTraceProvenanceV1 = { diff --git a/apps/sim/providers/runtime-context.test.ts b/apps/sim/providers/runtime-context.test.ts index de6bed10577..c2919d7f4bc 100644 --- a/apps/sim/providers/runtime-context.test.ts +++ b/apps/sim/providers/runtime-context.test.ts @@ -393,7 +393,7 @@ describe('provider runtime context', () => { expect(result.output).toBe('{{TOKEN}}') }) - it.each(['123', 'true'])( + it.each(['123'])( 'leaves non-model resource metadata untouched while projecting content (%s)', async (secret) => { const registry = new ResolvedSecretTraceRegistry([ From 5dbc342d7c2dcc2223ae5909bc4b8e1c04ed2798 Mon Sep 17 00:00:00 2001 From: Waleed Date: Mon, 10 Aug 2026 10:29:50 -0700 Subject: [PATCH 4/7] fix(provenance): name the importer that condemned a run (#6493) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(provenance): name the importer that condemned a run A bundle that arrives already incomplete latches the whole registry one-way, so every later model projection in the run refuses. #6483 made the refusal say what went wrong; it could not say who. In production the answer was reason=source-provenance-incomplete with 23 candidate importers and no way to tell them apart, which is where the last investigation stopped. Carry a stable origin across the import boundary and retain it beside the reason. It inherits through forks and merges — the step that erased attribution before, since a tool crossing forks, imports, then merges back — so the refusal names the importer even though the latch happened frames earlier. Tool crossings take the tool id, so a tool-sourced bundle identifies itself rather than being inferred from timestamps. Origins are caller-supplied strings rather than a closed union, so unlike reasons they carry an explicit bound. No behaviour change: the field is optional, additive, and read only when building a log record. * fix(provenance): attribute the path-scoped and short-circuit latches too importProvenanceForValueAtInputPath took only { trusted }, so the five callers that bind a crossing to an input path — the block, loop, parallel and workflow resolvers, and the guardrails route — could not name themselves. Separately, six latches sit beside a tagged import on the path where the import did not run or returned false: a bundle already marked incomplete short-circuits the || before the import, and each catch latches directly. Those reported no origin while their neighbour reported one. * fix(provenance): stop a broad catch claiming an incomplete bundle The catch around table-row provenance loading also covers a database failure in loadTableRowSecretProvenance, which is not an incomplete bundle. Naming a reason the catch cannot know is the misattribution this work exists to remove, so it reports 'unspecified' with its origin, matching every sibling catch. The decrypt catch keeps its specific reason because its try wraps only the decrypt call. --- .../app/api/guardrails/validate/route.test.ts | 1 + apps/sim/app/api/guardrails/validate/route.ts | 2 +- .../sim/app/api/mcp/serve/[serverId]/route.ts | 1 + apps/sim/app/api/providers/route.test.ts | 5 +- apps/sim/app/api/providers/route.ts | 2 +- apps/sim/app/api/workflows/[id]/log/route.ts | 5 +- .../mothership/mothership-handler.test.ts | 8 +- .../handlers/mothership/mothership-handler.ts | 5 +- .../handlers/workflow/workflow-handler.ts | 2 + ...resolved-secret-projection-refusal.test.ts | 29 ++++++ .../resolved-secret-projection-refusal.ts | 2 + .../resolved-secret-trace-registry.test.ts | 34 +++++++ .../utils/resolved-secret-trace-registry.ts | 91 ++++++++++++++----- .../sim/executor/variables/resolvers/block.ts | 2 +- apps/sim/executor/variables/resolvers/loop.ts | 2 +- .../executor/variables/resolvers/parallel.ts | 2 +- .../executor/variables/resolvers/workflow.ts | 2 +- apps/sim/lib/copilot/request/tools/client.ts | 7 +- .../tools/handlers/function-execute.ts | 19 +++- .../tools/handlers/workflow/mutations.ts | 4 +- .../execution/durable-secret-provenance.ts | 10 +- .../lib/guardrails/validate_hallucination.ts | 1 + .../lib/knowledge/model-input-provenance.ts | 1 + .../sim/lib/logs/execution/logging-session.ts | 5 +- apps/sim/lib/logs/execution/trace-store.ts | 12 ++- apps/sim/lib/table/backfill-runner.ts | 5 +- apps/sim/lib/table/rows/secret-provenance.ts | 8 +- .../workspace-file-secret-provenance.test.ts | 10 +- .../lib/workflows/executor/execution-core.ts | 2 +- .../executor/input-secret-provenance.ts | 5 +- apps/sim/tools/index.ts | 21 +++-- 31 files changed, 240 insertions(+), 65 deletions(-) diff --git a/apps/sim/app/api/guardrails/validate/route.test.ts b/apps/sim/app/api/guardrails/validate/route.test.ts index 5ae3dbe996a..797e7852af0 100644 --- a/apps/sim/app/api/guardrails/validate/route.test.ts +++ b/apps/sim/app/api/guardrails/validate/route.test.ts @@ -244,6 +244,7 @@ describe('POST /api/guardrails/validate', () => { expect(res.status).toBe(200) expect(mockImportProvenance).toHaveBeenCalledWith(provenance, 'secret value', ['input'], { trusted: true, + origin: 'guardrailsRoute.inputProvenance', }) }) diff --git a/apps/sim/app/api/guardrails/validate/route.ts b/apps/sim/app/api/guardrails/validate/route.ts index f0e50dcfa9d..d6edea5b4ef 100644 --- a/apps/sim/app/api/guardrails/validate/route.ts +++ b/apps/sim/app/api/guardrails/validate/route.ts @@ -267,7 +267,7 @@ export const POST = withRouteHandler(async (request: NextRequest) => { provenanceInspection.value, inputStr, ['input'], - { trusted: true } + { trusted: true, origin: 'guardrailsRoute.inputProvenance' } ) ).success : true diff --git a/apps/sim/app/api/mcp/serve/[serverId]/route.ts b/apps/sim/app/api/mcp/serve/[serverId]/route.ts index 211edf1305b..ee7cea60c34 100644 --- a/apps/sim/app/api/mcp/serve/[serverId]/route.ts +++ b/apps/sim/app/api/mcp/serve/[serverId]/route.ts @@ -288,6 +288,7 @@ async function projectWorkflowMcpModelContent( ): Promise { const registry = new ResolvedSecretTraceRegistry([], scope) const imported = await registry.importCrossingProvenance(privateProvenance, value, { + origin: 'mcpServe.workflowCrossing', trusted: true, }) if (!imported || !registry.isComplete()) { diff --git a/apps/sim/app/api/providers/route.test.ts b/apps/sim/app/api/providers/route.test.ts index 55641053694..e42cb93a08e 100644 --- a/apps/sim/app/api/providers/route.test.ts +++ b/apps/sim/app/api/providers/route.test.ts @@ -208,7 +208,10 @@ describe('POST /api/providers', () => { ) expect(res.status).toBe(200) - expect(mockImportProvenance).toHaveBeenCalledWith(provenance, { trusted: true }) + expect(mockImportProvenance).toHaveBeenCalledWith(provenance, { + trusted: true, + origin: 'providersRoute.requestProvenance', + }) }) it('projects legacy private prompt provenance on the provider-facing copy', async () => { diff --git a/apps/sim/app/api/providers/route.ts b/apps/sim/app/api/providers/route.ts index 2fe21daae23..7e6d9845381 100644 --- a/apps/sim/app/api/providers/route.ts +++ b/apps/sim/app/api/providers/route.ts @@ -272,7 +272,7 @@ export const POST = withRouteHandler(async (request: NextRequest) => { const provenanceReady = await providerRuntimeContext.resolvedSecretTraceRegistry.importProvenance( provenanceInspection.value, - { trusted: true } + { trusted: true, origin: 'providersRoute.requestProvenance' } ) if (!provenanceReady || !providerRuntimeContext.resolvedSecretTraceRegistry.isComplete()) { return NextResponse.json( diff --git a/apps/sim/app/api/workflows/[id]/log/route.ts b/apps/sim/app/api/workflows/[id]/log/route.ts index 8e5123504ed..93984773999 100644 --- a/apps/sim/app/api/workflows/[id]/log/route.ts +++ b/apps/sim/app/api/workflows/[id]/log/route.ts @@ -144,7 +144,10 @@ export const POST = withRouteHandler( if (trustedProvenance === undefined) { resolvedSecretTraceRegistry.markIncomplete() } else { - await resolvedSecretTraceRegistry.importProvenance(trustedProvenance, { trusted: true }) + await resolvedSecretTraceRegistry.importProvenance(trustedProvenance, { + trusted: true, + origin: 'workflowLogRoute.trustedProvenance', + }) } loggingSession.setResolvedSecretTraceRegistry(resolvedSecretTraceRegistry) diff --git a/apps/sim/executor/handlers/mothership/mothership-handler.test.ts b/apps/sim/executor/handlers/mothership/mothership-handler.test.ts index 6c9a752ec05..aa211829f4f 100644 --- a/apps/sim/executor/handlers/mothership/mothership-handler.test.ts +++ b/apps/sim/executor/handlers/mothership/mothership-handler.test.ts @@ -293,7 +293,7 @@ describe('MothershipBlockHandler', () => { content: 'raw secret remains functional', __resolvedSecretTraceProvenance: undefined, }), - { trusted: true } + { trusted: true, origin: 'mothership.payloadCrossing' } ) expect(registry.markIncomplete).not.toHaveBeenCalled() expect(result).toMatchObject({ content: 'raw secret remains functional' }) @@ -519,7 +519,7 @@ describe('MothershipBlockHandler', () => { error: 'secret-backed failure', __resolvedSecretTraceProvenance: undefined, }), - { trusted: true } + { trusted: true, origin: 'mothership.payloadCrossing' } ) expect(context.errorResolvedSecretTraceRegistry).toBeDefined() expect(context.errorResolvedSecretTraceRegistry).not.toBe(context.resolvedSecretTraceRegistry) @@ -557,7 +557,7 @@ describe('MothershipBlockHandler', () => { error: 'secret-backed failure', __resolvedSecretTraceProvenance: undefined, }), - { trusted: true } + { trusted: true, origin: 'mothership.payloadCrossing' } ) expect(registry.markIncomplete).not.toHaveBeenCalled() expect(context.errorResolvedSecretTraceRegistry).toBeDefined() @@ -601,7 +601,7 @@ describe('MothershipBlockHandler', () => { content: 'unchanged', __resolvedSecretTraceProvenance: undefined, }), - { trusted: true } + { trusted: true, origin: 'mothership.payloadCrossing' } ) expect(registry.markIncomplete).not.toHaveBeenCalled() expect(JSON.stringify(result.execution.output)).not.toContain('__resolvedSecretTraceProvenance') diff --git a/apps/sim/executor/handlers/mothership/mothership-handler.ts b/apps/sim/executor/handlers/mothership/mothership-handler.ts index b3af3d828d5..1d285cd5df3 100644 --- a/apps/sim/executor/handlers/mothership/mothership-handler.ts +++ b/apps/sim/executor/handlers/mothership/mothership-handler.ts @@ -378,7 +378,10 @@ async function consumeMothershipProvenance( if (!registry) return false - const imported = await registry.importProvenanceForValue(provenance, payload, { trusted: true }) + const imported = await registry.importProvenanceForValue(provenance, payload, { + trusted: true, + origin: 'mothership.payloadCrossing', + }) if (!imported) throw new Error('Mothership response provenance metadata is invalid') return true } diff --git a/apps/sim/executor/handlers/workflow/workflow-handler.ts b/apps/sim/executor/handlers/workflow/workflow-handler.ts index e795c2b99e9..73aa5d4fbf7 100644 --- a/apps/sim/executor/handlers/workflow/workflow-handler.ts +++ b/apps/sim/executor/handlers/workflow/workflow-handler.ts @@ -450,6 +450,7 @@ export class WorkflowBlockHandler implements BlockHandler { await childResolvedSecretTraceRegistry.importProvenance(crossingProvenance, { trusted: true, anonymous: true, + origin: 'workflowHandler.childCrossing', }) } // Custom-block children authenticate internal tool calls as the source @@ -711,6 +712,7 @@ export class WorkflowBlockHandler implements BlockHandler { await ctx.resolvedSecretTraceRegistry.importProvenance(crossingProvenance, { trusted: true, anonymous: true, + origin: 'workflowHandler.parentCrossing', }) } return exposedOutput diff --git a/apps/sim/executor/utils/resolved-secret-projection-refusal.test.ts b/apps/sim/executor/utils/resolved-secret-projection-refusal.test.ts index 17cdfef8d63..16924a314dd 100644 --- a/apps/sim/executor/utils/resolved-secret-projection-refusal.test.ts +++ b/apps/sim/executor/utils/resolved-secret-projection-refusal.test.ts @@ -195,6 +195,35 @@ describe('refuseResolvedSecretProjection', () => { expect(refusalRecords()).toHaveLength(3) }) + it('names the importer that condemned the run, through the fork and merge that hid it', async () => { + const parent = new ResolvedSecretTraceRegistry([], scope) + const fork = parent.forkForToolCall() + await fork.importCrossingProvenance( + { version: 1, complete: false, entries: [], scope }, + { rows: [] }, + { trusted: true, origin: 'tool.table_query_rows' } + ) + parent.mergeToolCallRegistry(fork) + mockLogger.error.mockClear() + mockLogger.warn.mockClear() + + expect(() => + refuseResolvedSecretProjection({ + site: 'router.contextModelInput', + message: 'Router model input could not be safely projected', + registry: parent, + inputPath: 'context,routes', + }) + ).toThrow() + + expect(refusalRecords()[0][1]).toEqual( + expect.objectContaining({ + reason: 'source-provenance-incomplete', + origins: ['tool.table_query_rows'], + }) + ) + }) + it('records no secret material', () => { const registry = new ResolvedSecretTraceRegistry( [{ name: 'API_KEY', plaintext: 'super-secret-value', encryptedValue: 'encrypted' }], diff --git a/apps/sim/executor/utils/resolved-secret-projection-refusal.ts b/apps/sim/executor/utils/resolved-secret-projection-refusal.ts index bd8837d7245..24a573225ea 100644 --- a/apps/sim/executor/utils/resolved-secret-projection-refusal.ts +++ b/apps/sim/executor/utils/resolved-secret-projection-refusal.ts @@ -64,6 +64,8 @@ function reportRefusal({ site, registry, inputPath }: ResolvedSecretProjectionRe ? { reason: diagnostics.reasons[0], reasons: diagnostics.reasons, + /** The importers that cost the run its completeness — who to go and look at. */ + ...(diagnostics.origins.length > 0 ? { origins: diagnostics.origins } : {}), incompleteInputPathCount: diagnostics.incompleteInputPathCount, activeEntryCount: diagnostics.activeEntryCount, ...(diagnostics.scopeWorkspaceId diff --git a/apps/sim/executor/utils/resolved-secret-trace-registry.test.ts b/apps/sim/executor/utils/resolved-secret-trace-registry.test.ts index 6c2f057e2fb..e16030c2d88 100644 --- a/apps/sim/executor/utils/resolved-secret-trace-registry.test.ts +++ b/apps/sim/executor/utils/resolved-secret-trace-registry.test.ts @@ -1562,6 +1562,40 @@ describe('incompleteness diagnostics', () => { expect(registry.getIncompletenessDiagnostics()?.reasons[0]).toBe('constructed-incomplete') }) + it('attributes an untrustworthy bundle to the caller that imported it', async () => { + const registry = new ResolvedSecretTraceRegistry([], scope) + + await registry.importProvenance( + { version: 1, complete: false, entries: [], scope }, + { trusted: true, origin: 'workflowHandler.childCrossing' } + ) + + expect(registry.getIncompletenessDiagnostics()?.origins).toEqual([ + 'workflowHandler.childCrossing', + ]) + expect(mockLogger.warn).toHaveBeenCalledWith( + 'Resolved secret registry marked incomplete', + expect.objectContaining({ + reason: 'source-provenance-incomplete', + origin: 'workflowHandler.childCrossing', + }) + ) + }) + + it('bounds retained origins, which are caller-supplied rather than a closed union', async () => { + const registry = new ResolvedSecretTraceRegistry([], scope) + + for (let index = 0; index < 20; index++) { + await registry.importProvenance( + { version: 1, complete: false, entries: [], scope }, + { trusted: true, origin: `caller.${index}` } + ) + } + + expect(registry.getIncompletenessDiagnostics()?.origins).toHaveLength(8) + expect(registry.getIncompletenessDiagnostics()?.origins[0]).toBe('caller.0') + }) + it('records no secret material alongside the reason', () => { const registry = new ResolvedSecretTraceRegistry([], scope) diff --git a/apps/sim/executor/utils/resolved-secret-trace-registry.ts b/apps/sim/executor/utils/resolved-secret-trace-registry.ts index 377c89d72da..c979642edfa 100644 --- a/apps/sim/executor/utils/resolved-secret-trace-registry.ts +++ b/apps/sim/executor/utils/resolved-secret-trace-registry.ts @@ -79,6 +79,12 @@ const BY_DESIGN_INCOMPLETENESS_REASONS = new Set } +/** Extra attribution for a latch: which registry it propagated from, and which importer caused it. */ +interface MarkIncompleteContext { + source?: ResolvedSecretTraceRegistry + origin?: string +} + export interface ImportResolvedSecretTraceProvenanceOptions { trusted: boolean anonymous?: boolean + /** + * Stable dotted identifier for the caller, e.g. `workflowHandler.childCrossing`. + * + * A bundle that arrives already incomplete condemns the whole run, and the reason alone cannot + * say which of the many importers accepted it. Recording the caller is what turns + * `source-provenance-incomplete` from a symptom into an address. + */ + origin?: string } export interface ExportResolvedSecretTraceProvenanceForValueOptions { @@ -627,6 +649,8 @@ export class ResolvedSecretTraceRegistry { private readonly incompleteInputPaths = new Map() /** Insertion-ordered; see {@link ResolvedSecretIncompletenessDiagnostics}. */ private readonly incompletenessReasons = new Set() + /** Import callers that cost this registry its completeness; bounded by {@link MAX_RETAINED_ORIGINS}. */ + private readonly incompletenessOrigins = new Set() private activeProvenanceEntryBytes = 0 private complete = true private pendingActivations = 0 @@ -681,7 +705,7 @@ export class ResolvedSecretTraceRegistry { } this.copyResolvedInputPathsTo(fork) this.copyIncompleteInputPathsTo(fork) - if (!this.complete) fork.markIncomplete('inherited-incomplete-source', this) + if (!this.complete) fork.markIncomplete('inherited-incomplete-source', { source: this }) return fork } @@ -692,12 +716,12 @@ export class ResolvedSecretTraceRegistry { ): ResolvedSecretTraceRegistry { const fork = new ResolvedSecretTraceRegistry(this.catalog.values(), this.scope) if (!this.complete) { - fork.markIncomplete('inherited-incomplete-source', this) + fork.markIncomplete('inherited-incomplete-source', { source: this }) return fork } if (this.hasIncompleteInputPathOverlapping(paths)) { - fork.markIncomplete('inherited-incomplete-input-path', this) + fork.markIncomplete('inherited-incomplete-input-path', { source: this }) return fork } @@ -721,7 +745,8 @@ export class ResolvedSecretTraceRegistry { fork.addActiveEntry({ ...entry }, { propagated: true }) } } - if (this.isPermanentlyIncomplete()) fork.markIncomplete('inherited-incomplete-source', this) + if (this.isPermanentlyIncomplete()) + fork.markIncomplete('inherited-incomplete-source', { source: this }) return fork } @@ -733,7 +758,7 @@ export class ResolvedSecretTraceRegistry { } if (!child.isComplete()) { - this.markIncomplete('inherited-incomplete-source', child) + this.markIncomplete('inherited-incomplete-source', { source: child }) return } @@ -1100,12 +1125,12 @@ export class ResolvedSecretTraceRegistry { options: ImportResolvedSecretTraceProvenanceOptions ): Promise { if (!options.trusted || !isResolvedSecretTraceProvenanceV1(provenance)) { - this.markIncomplete('untrusted-provenance') + this.markIncomplete('untrusted-provenance', { origin: options.origin }) return false } if (!provenance.complete) { - this.markIncomplete('source-provenance-incomplete') + this.markIncomplete('source-provenance-incomplete', { origin: options.origin }) } const sameScope = scopesMatch(provenance.scope, this.scope) @@ -1128,7 +1153,7 @@ export class ResolvedSecretTraceRegistry { importedAll = false decryptFailures += 1 firstDecryptError ??= getErrorMessage(error, 'Unknown error') - this.markIncomplete('entry-decrypt-failed') + this.markIncomplete('entry-decrypt-failed', { origin: options.origin }) } } @@ -1156,7 +1181,7 @@ export class ResolvedSecretTraceRegistry { async importProvenanceForValue( provenance: unknown, value: unknown, - options: { trusted: boolean } + options: { trusted: boolean; origin?: string } ): Promise { const result = await this.importProvenanceForValueInternal(provenance, value, options) return result.success @@ -1167,7 +1192,7 @@ export class ResolvedSecretTraceRegistry { provenance: unknown, value: unknown, inputPath: ResolvedSecretInputPath | undefined, - options: { trusted: boolean } + options: { trusted: boolean; origin?: string } ): Promise { return this.importProvenanceForValueInternal(provenance, value, { ...options, @@ -1178,28 +1203,39 @@ export class ResolvedSecretTraceRegistry { private async importProvenanceForValueInternal( provenance: unknown, value: unknown, - options: { trusted: boolean; inputPath?: ResolvedSecretInputPath } + options: { trusted: boolean; inputPath?: ResolvedSecretInputPath; origin?: string } ): Promise { if (!options.trusted || !isResolvedSecretTraceProvenanceV1(provenance)) { - this.markInputPathIncomplete(options.inputPath, 'value-provenance-untrusted') + this.markInputPathIncomplete(options.inputPath, 'value-provenance-untrusted', options.origin) return { success: false, matched: false } } const sourceRegistry = new ResolvedSecretTraceRegistry([], provenance.scope, { staged: true }) - const sourceImported = await sourceRegistry.importProvenance(provenance, { trusted: true }) + const sourceImported = await sourceRegistry.importProvenance(provenance, { + trusted: true, + origin: options.origin, + }) const filteredProvenance = sourceRegistry.exportProvenanceForValue(value) if (!sourceImported) { - this.markInputPathIncomplete(options.inputPath, 'value-provenance-import-failed') + this.markInputPathIncomplete( + options.inputPath, + 'value-provenance-import-failed', + options.origin + ) return { success: false, matched: false } } if (!filteredProvenance.complete) { this.markInputPathIncomplete( options.inputPath, - provenance.complete ? 'value-provenance-filter-incomplete' : 'source-provenance-incomplete' + provenance.complete ? 'value-provenance-filter-incomplete' : 'source-provenance-incomplete', + options.origin ) return { success: true, matched: false } } - const filteredImported = await this.importProvenance(filteredProvenance, { trusted: true }) + const filteredImported = await this.importProvenance(filteredProvenance, { + trusted: true, + origin: options.origin, + }) if (options.inputPath && options.inputPath.length > 0 && filteredProvenance.complete) { const sameScope = scopesMatch(filteredProvenance.scope, this.scope) this.bindResolvedInputPathEntries( @@ -1228,7 +1264,7 @@ export class ResolvedSecretTraceRegistry { async importCrossingProvenance( provenance: unknown, crossingValue: unknown, - options: { trusted: boolean } + options: { trusted: boolean; origin?: string } ): Promise { return this.importProvenanceForValue(provenance, crossingValue, options) } @@ -1351,6 +1387,7 @@ export class ResolvedSecretTraceRegistry { if (!this.isPermanentlyIncomplete()) return undefined return { reasons: [...this.incompletenessReasons], + origins: [...this.incompletenessOrigins], incompleteInputPathCount: this.incompleteInputPaths.size, activeEntryCount: this.activeEntries.size, ...(this.scope?.workspaceId ? { scopeWorkspaceId: this.scope.workspaceId } : {}), @@ -1362,12 +1399,19 @@ export class ResolvedSecretTraceRegistry { this.incompletenessReasons.add(reason) } + /** Retains the importing caller, keeping the earliest once the bound is reached. */ + private recordIncompletenessOrigin(origin: string): void { + if (this.incompletenessOrigins.size >= MAX_RETAINED_ORIGINS) return + this.incompletenessOrigins.add(origin) + } + /** * Carries a source registry's reasons into a fork or merge target, so a refusal downstream still * names the guard that originally tripped rather than only the propagation that reached it. */ private inheritIncompletenessReasonsFrom(source: ResolvedSecretTraceRegistry): void { for (const reason of source.incompletenessReasons) this.recordIncompletenessReason(reason) + for (const origin of source.incompletenessOrigins) this.recordIncompletenessOrigin(origin) } isPermanentlyIncomplete(): boolean { @@ -1376,16 +1420,18 @@ export class ResolvedSecretTraceRegistry { markIncomplete( reason: ResolvedSecretIncompletenessReason = 'unspecified', - source?: ResolvedSecretTraceRegistry + context: MarkIncompleteContext = {} ): void { - if (source) this.inheritIncompletenessReasonsFrom(source) + if (context.source) this.inheritIncompletenessReasonsFrom(context.source) this.recordIncompletenessReason(reason) + if (context.origin) this.recordIncompletenessOrigin(context.origin) if (!this.complete) return this.complete = false this.modelEgressRevision += 1 if (this.staged || BY_DESIGN_INCOMPLETENESS_REASONS.has(reason)) return const details = { reason, + ...(context.origin ? { origin: context.origin } : {}), scopeWorkspaceId: this.scope?.workspaceId, activeEntryCount: this.activeEntries.size, incompleteInputPathCount: this.incompleteInputPaths.size, @@ -1824,13 +1870,15 @@ export class ResolvedSecretTraceRegistry { private markInputPathIncomplete( path: ResolvedSecretInputPath | undefined, - reason: ResolvedSecretIncompletenessReason = 'unspecified' + reason: ResolvedSecretIncompletenessReason = 'unspecified', + origin?: string ): void { if (!path || path.length === 0) { - this.markIncomplete(reason) + this.markIncomplete(reason, { origin }) return } this.recordIncompletenessReason(reason) + if (origin) this.recordIncompletenessOrigin(origin) const key = inputPathKey(path) if (this.incompleteInputPaths.has(key)) return this.incompleteInputPaths.set(key, [...path]) @@ -1838,6 +1886,7 @@ export class ResolvedSecretTraceRegistry { if (this.staged || BY_DESIGN_INCOMPLETENESS_REASONS.has(reason)) return const details = { reason, + ...(origin ? { origin } : {}), inputPath: path.join('.'), scopeWorkspaceId: this.scope?.workspaceId, activeEntryCount: this.activeEntries.size, diff --git a/apps/sim/executor/variables/resolvers/block.ts b/apps/sim/executor/variables/resolvers/block.ts index b057a97501c..83f20eecd27 100644 --- a/apps/sim/executor/variables/resolvers/block.ts +++ b/apps/sim/executor/variables/resolvers/block.ts @@ -337,7 +337,7 @@ export class BlockResolver implements Resolver { state.resolvedSecretTraceProvenance, value, context.inputPath, - { trusted: true } + { trusted: true, origin: 'blockResolver.outputCrossing' } ) if (imported.matched) context.onResolvedSecretReference?.() return value diff --git a/apps/sim/executor/variables/resolvers/loop.ts b/apps/sim/executor/variables/resolvers/loop.ts index 3c29af6a45c..523af3cfc9f 100644 --- a/apps/sim/executor/variables/resolvers/loop.ts +++ b/apps/sim/executor/variables/resolvers/loop.ts @@ -283,7 +283,7 @@ export class LoopResolver implements Resolver { provenance, resolvedValue, context.inputPath, - { trusted: true } + { trusted: true, origin: 'loopResolver.itemCrossing' } ) if (imported.matched) context.onResolvedSecretReference?.() return resolvedValue diff --git a/apps/sim/executor/variables/resolvers/parallel.ts b/apps/sim/executor/variables/resolvers/parallel.ts index fdec9d5cdd0..12c95cf9ca8 100644 --- a/apps/sim/executor/variables/resolvers/parallel.ts +++ b/apps/sim/executor/variables/resolvers/parallel.ts @@ -390,7 +390,7 @@ export class ParallelResolver implements Resolver { provenance, resolvedValue, context.inputPath, - { trusted: true } + { trusted: true, origin: 'parallelResolver.itemCrossing' } ) if (imported.matched) context.onResolvedSecretReference?.() return resolvedValue diff --git a/apps/sim/executor/variables/resolvers/workflow.ts b/apps/sim/executor/variables/resolvers/workflow.ts index 7bc0a6299ad..8d0c778016b 100644 --- a/apps/sim/executor/variables/resolvers/workflow.ts +++ b/apps/sim/executor/variables/resolvers/workflow.ts @@ -133,7 +133,7 @@ export class WorkflowResolver implements Resolver { provenance, value, context.inputPath, - { trusted: true } + { trusted: true, origin: 'workflowResolver.inputCrossing' } ) if (imported.matched) context.onResolvedSecretReference?.() return value diff --git a/apps/sim/lib/copilot/request/tools/client.ts b/apps/sim/lib/copilot/request/tools/client.ts index a7aac65cd12..c2651bfa114 100644 --- a/apps/sim/lib/copilot/request/tools/client.ts +++ b/apps/sim/lib/copilot/request/tools/client.ts @@ -103,17 +103,20 @@ export async function waitForClientToolCompletion({ toolRegistry.markIncomplete() } else { const imported = await toolRegistry.importProvenance(sealedContext.provenance, { + origin: 'copilotToolClient.sealedContext', trusted: true, }) if (!imported || !sealedContext.provenance.complete) { - toolRegistry.markIncomplete() + toolRegistry.markIncomplete('source-provenance-incomplete', { + origin: 'copilotToolClient.sealedContext', + }) } else { content = sealedContent } } } } catch { - toolRegistry?.markIncomplete() + toolRegistry?.markIncomplete('unspecified', { origin: 'copilotToolClient.sealedContext' }) } finally { finishPendingActivation?.() } diff --git a/apps/sim/lib/copilot/tools/handlers/function-execute.ts b/apps/sim/lib/copilot/tools/handlers/function-execute.ts index da324d58906..d6f04a63c33 100644 --- a/apps/sim/lib/copilot/tools/handlers/function-execute.ts +++ b/apps/sim/lib/copilot/tools/handlers/function-execute.ts @@ -528,12 +528,19 @@ export async function resolveInputFiles( }) if ( !provenance.complete || - !(await resolvedSecretTraceRegistry.importProvenance(provenance, { trusted: true })) + !(await resolvedSecretTraceRegistry.importProvenance(provenance, { + trusted: true, + origin: 'copilotFunctionExecute.result', + })) ) { - resolvedSecretTraceRegistry.markIncomplete() + resolvedSecretTraceRegistry.markIncomplete('source-provenance-incomplete', { + origin: 'copilotFunctionExecute.result', + }) } } catch { - resolvedSecretTraceRegistry.markIncomplete() + resolvedSecretTraceRegistry.markIncomplete('unspecified', { + origin: 'copilotFunctionExecute.result', + }) } const columns = table.schema.columns @@ -561,11 +568,13 @@ async function importMountedProvenance( try { const provenance = source.exportProvenanceForValue(crossingValue) const imported = await target.importCrossingProvenance(provenance, crossingValue, { + origin: 'copilotFunctionExecute.crossing', trusted: true, }) - if (!imported) target.markIncomplete() + if (!imported) + target.markIncomplete('unspecified', { origin: 'copilotFunctionExecute.crossing' }) } catch { - target.markIncomplete() + target.markIncomplete('unspecified', { origin: 'copilotFunctionExecute.crossing' }) } } diff --git a/apps/sim/lib/copilot/tools/handlers/workflow/mutations.ts b/apps/sim/lib/copilot/tools/handlers/workflow/mutations.ts index 260bf73fbd2..dc74ef922ae 100644 --- a/apps/sim/lib/copilot/tools/handlers/workflow/mutations.ts +++ b/apps/sim/lib/copilot/tools/handlers/workflow/mutations.ts @@ -119,7 +119,7 @@ async function executeCopilotWorkflowTarget(params: { await params.context.resolvedSecretTraceRegistry.importCrossingProvenance( result.executionState?.resolvedSecretTraceProvenance, { output: result.output, logs: result.logs, error: result.error }, - { trusted: true } + { trusted: true, origin: 'copilotWorkflowMutation.runCrossing' } ) } return result @@ -134,7 +134,7 @@ async function executeCopilotWorkflowTarget(params: { error: executionResult?.error, thrownMessage: toError(error).message, }, - { trusted: true } + { trusted: true, origin: 'copilotWorkflowMutation.failedRunCrossing' } ) } if (admission.targetReservation) { diff --git a/apps/sim/lib/execution/durable-secret-provenance.ts b/apps/sim/lib/execution/durable-secret-provenance.ts index edd4a1b7df8..724f0ac340d 100644 --- a/apps/sim/lib/execution/durable-secret-provenance.ts +++ b/apps/sim/lib/execution/durable-secret-provenance.ts @@ -256,8 +256,14 @@ export async function importDurableSecretProvenance( } const imported = value === undefined - ? await registry.importProvenance(envelope, { trusted: true }) - : await registry.importProvenanceForValue(envelope, value, { trusted: true }) + ? await registry.importProvenance(envelope, { + trusted: true, + origin: 'durableProvenance.envelope', + }) + : await registry.importProvenanceForValue(envelope, value, { + trusted: true, + origin: 'durableProvenance.valueEnvelope', + }) complete = imported && complete } return complete && !registry.isPermanentlyIncomplete() diff --git a/apps/sim/lib/guardrails/validate_hallucination.ts b/apps/sim/lib/guardrails/validate_hallucination.ts index 7185236baaf..155af832993 100644 --- a/apps/sim/lib/guardrails/validate_hallucination.ts +++ b/apps/sim/lib/guardrails/validate_hallucination.ts @@ -143,6 +143,7 @@ async function queryKnowledgeBase( functionalResponse = { ...payload } delete functionalResponse[RESOLVED_SECRET_PROVENANCE_FIELD] const imported = await resultRegistry.importProvenance(inspection.value, { + origin: 'guardrails.hallucinationResult', trusted: true, }) if (!imported || !resultRegistry.isComplete()) { diff --git a/apps/sim/lib/knowledge/model-input-provenance.ts b/apps/sim/lib/knowledge/model-input-provenance.ts index af4ea906476..30fbfe8f706 100644 --- a/apps/sim/lib/knowledge/model-input-provenance.ts +++ b/apps/sim/lib/knowledge/model-input-provenance.ts @@ -65,6 +65,7 @@ export async function prepareKnowledgeModelInputProvenance(options: { } const imported = await registry.importProvenanceForValue(inspection.value, options.modelInput, { + origin: 'knowledge.modelInputProvenance', trusted: true, }) if (!imported || !registry.isComplete()) { diff --git a/apps/sim/lib/logs/execution/logging-session.ts b/apps/sim/lib/logs/execution/logging-session.ts index d6b51552096..ced3101d10c 100644 --- a/apps/sim/lib/logs/execution/logging-session.ts +++ b/apps/sim/lib/logs/execution/logging-session.ts @@ -316,7 +316,10 @@ export class LoggingSession { } const registry = new ResolvedSecretTraceRegistry([], provenance.scope) - await registry.importProvenance(provenance, { trusted: true }) + await registry.importProvenance(provenance, { + trusted: true, + origin: 'loggingSession.restoredProvenance', + }) return registry } diff --git a/apps/sim/lib/logs/execution/trace-store.ts b/apps/sim/lib/logs/execution/trace-store.ts index 031d86216d0..9fa5a7b05df 100644 --- a/apps/sim/lib/logs/execution/trace-store.ts +++ b/apps/sim/lib/logs/execution/trace-store.ts @@ -275,7 +275,10 @@ export async function projectExecutionDataForDisplay( if (isResolvedSecretTraceProvenanceV1(provenance)) { registry = new ResolvedSecretTraceRegistry([], provenance.scope) - await registry.importProvenance(provenance, { trusted: true }) + await registry.importProvenance(provenance, { + trusted: true, + origin: 'traceStore.spanProvenance', + }) } const projectionStore = { @@ -301,9 +304,12 @@ export async function projectExecutionDataForDisplay( ? new ResolvedSecretTraceRegistry([], exactProvenance.scope) : new ResolvedSecretTraceRegistry() if (isResolvedSecretTraceProvenanceV1(exactProvenance)) { - await exactRegistry.importProvenance(exactProvenance, { trusted: true }) + await exactRegistry.importProvenance(exactProvenance, { + trusted: true, + origin: 'traceStore.exactProvenance', + }) } else { - exactRegistry.markIncomplete() + exactRegistry.markIncomplete('untrusted-provenance', { origin: 'traceStore.exactProvenance' }) } const [projected] = await projectTraceSpansForSecrets( diff --git a/apps/sim/lib/table/backfill-runner.ts b/apps/sim/lib/table/backfill-runner.ts index c9041f84b87..1d549397ed5 100644 --- a/apps/sim/lib/table/backfill-runner.ts +++ b/apps/sim/lib/table/backfill-runner.ts @@ -82,7 +82,10 @@ export async function createBackfillExecutionSecretRegistry(options: { registry.markIncomplete() return registry } - await registry.importProvenance(provenance, { trusted: true }) + await registry.importProvenance(provenance, { + trusted: true, + origin: 'tableBackfill.rowProvenance', + }) return registry } diff --git a/apps/sim/lib/table/rows/secret-provenance.ts b/apps/sim/lib/table/rows/secret-provenance.ts index 160bffa73c1..7ea7291107f 100644 --- a/apps/sim/lib/table/rows/secret-provenance.ts +++ b/apps/sim/lib/table/rows/secret-provenance.ts @@ -256,7 +256,13 @@ export async function createTableRowSecretProvenanceFromEncryptedExecution( return createUnknownTableRowSecretProvenance() } const registry = new ResolvedSecretTraceRegistry([], provenance.scope) - if (!(await registry.importProvenance(provenance, { trusted: true })) || !registry.isComplete()) { + if ( + !(await registry.importProvenance(provenance, { + trusted: true, + origin: 'tableRow.encryptedExecution', + })) || + !registry.isComplete() + ) { return createUnknownTableRowSecretProvenance() } return createTableRowSecretProvenanceFromRegistry(data, registry) diff --git a/apps/sim/lib/uploads/contexts/workspace/workspace-file-secret-provenance.test.ts b/apps/sim/lib/uploads/contexts/workspace/workspace-file-secret-provenance.test.ts index a565b08510b..124c66e0337 100644 --- a/apps/sim/lib/uploads/contexts/workspace/workspace-file-secret-provenance.test.ts +++ b/apps/sim/lib/uploads/contexts/workspace/workspace-file-secret-provenance.test.ts @@ -450,7 +450,7 @@ describe('workspace file secret provenance', () => { ], scope: { userId: 'user-1' }, }, - { trusted: true } + { trusted: true, origin: 'durableProvenance.envelope' } ) queueTableRows(workspaceFiles, [ @@ -488,7 +488,7 @@ describe('workspace file secret provenance', () => { ], scope: { userId: 'user-1' }, }, - { trusted: true } + { trusted: true, origin: 'durableProvenance.envelope' } ) queueTableRows(workspaceFiles, [ @@ -523,7 +523,7 @@ describe('workspace file secret provenance', () => { entries: [{ encryptedValue: 'anonymous-encrypted' }], scope: { userId: 'user-1' }, }, - { trusted: true } + { trusted: true, origin: 'durableProvenance.envelope' } ) queueTableRows(workspaceFiles, [ @@ -583,7 +583,7 @@ describe('workspace file secret provenance', () => { ], scope: { userId: 'user-1' }, }, - { trusted: true } + { trusted: true, origin: 'durableProvenance.envelope' } ) }) @@ -669,7 +669,7 @@ describe('workspace file secret provenance', () => { scope: { userId: 'user-1' }, }, 'derived text', - { trusted: true } + { trusted: true, origin: 'durableProvenance.valueEnvelope' } ) }) diff --git a/apps/sim/lib/workflows/executor/execution-core.ts b/apps/sim/lib/workflows/executor/execution-core.ts index 90af6185620..aa1fd9094f5 100644 --- a/apps/sim/lib/workflows/executor/execution-core.ts +++ b/apps/sim/lib/workflows/executor/execution-core.ts @@ -544,7 +544,7 @@ async function executeWorkflowCoreImpl( if (options.trustedInitialResolvedSecretTraceProvenance !== undefined) { await resolvedSecretTraceRegistry.importProvenance( options.trustedInitialResolvedSecretTraceProvenance, - { trusted: true } + { trusted: true, origin: 'executionCore.initialProvenance' } ) } loggingSession.setResolvedSecretTraceRegistry(resolvedSecretTraceRegistry) diff --git a/apps/sim/lib/workflows/executor/input-secret-provenance.ts b/apps/sim/lib/workflows/executor/input-secret-provenance.ts index bc93c6f4ec4..ecfbc83e1ed 100644 --- a/apps/sim/lib/workflows/executor/input-secret-provenance.ts +++ b/apps/sim/lib/workflows/executor/input-secret-provenance.ts @@ -85,7 +85,10 @@ export async function resolveWorkflowInputSecretProvenance(options: { } const sourceRegistry = new ResolvedSecretTraceRegistry([], provenance.scope) - const imported = await sourceRegistry.importProvenance(provenance, { trusted: true }) + const imported = await sourceRegistry.importProvenance(provenance, { + trusted: true, + origin: 'executionInput.secretProvenance', + }) const inputProvenance = sourceRegistry.exportProvenanceForValue(options.input) if ( !imported || diff --git a/apps/sim/tools/index.ts b/apps/sim/tools/index.ts index 22981884f81..6e36e73d284 100644 --- a/apps/sim/tools/index.ts +++ b/apps/sim/tools/index.ts @@ -1187,7 +1187,8 @@ function consumeResolvedSecretNames( async function consumeResolvedSecretProvenance( payload: unknown, - registry?: ResolvedSecretTraceRegistry + registry: ResolvedSecretTraceRegistry | undefined, + toolId: string ): Promise { if (!payload || typeof payload !== 'object' || Array.isArray(payload)) return false @@ -1202,6 +1203,7 @@ async function consumeResolvedSecretProvenance( const targetRegistry = registry.forkForToolCall() const imported = await targetRegistry.importCrossingProvenance(provenance, response, { trusted: true, + origin: `tool.${toolId}`, }) if (!imported) return false registry.mergeToolCallRegistry(targetRegistry) @@ -1249,7 +1251,8 @@ async function consumePrivateToolPayloadMetadata( headers: Headers, requestedType: PrivateToolMetadataType | undefined, params: Record, - registry?: ResolvedSecretTraceRegistry + registry: ResolvedSecretTraceRegistry | undefined, + toolId: string ): Promise { if (!requestedType) return 'verified' @@ -1308,7 +1311,7 @@ async function consumePrivateToolPayloadMetadata( ) { if (!consumeResolvedSecretNames(record, params, registry)) return 'invalid' } else { - if (!(await consumeResolvedSecretProvenance(record, registry))) return 'invalid' + if (!(await consumeResolvedSecretProvenance(record, registry, toolId))) return 'invalid' } } catch { return 'invalid' @@ -1323,7 +1326,8 @@ async function consumePrivateToolResponseMetadata( response: Response, requestedType: PrivateToolMetadataType | undefined, params: Record, - registry?: ResolvedSecretTraceRegistry + registry: ResolvedSecretTraceRegistry | undefined, + toolId: string ): Promise { if (!requestedType) return { response } @@ -1347,7 +1351,8 @@ async function consumePrivateToolResponseMetadata( response.headers, requestedType, params, - registry + registry, + toolId ) if (consumption === 'invalid') { return { response: rebuildSafePrivateToolResponse(response) } @@ -2512,7 +2517,8 @@ async function executeToolRequest( response, privateToolMetadataType, params, - resolvedSecretTraceRegistry + resolvedSecretTraceRegistry, + toolId ) response = privateMetadata.response @@ -2938,7 +2944,8 @@ async function executeMcpTool( response, privateToolMetadataType, params, - resolvedSecretTraceRegistry + resolvedSecretTraceRegistry, + toolId ) ).response From 5da48d015ab13677813dfd49402e40f814bdc88e Mon Sep 17 00:00:00 2001 From: Waleed Date: Mon, 10 Aug 2026 11:01:40 -0700 Subject: [PATCH 5/7] fix(env): assign window.__ENV directly instead of queueing it (#6494) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `` defaults to Next's `