Honor in-app blocking settings by default in AI Guard evaluate#10818
Open
Honor in-app blocking settings by default in AI Guard evaluate#10818
Conversation
Change Options.block default from false to true so that evaluate() follows the remote is_blocking_enabled setting without requiring explicit opt-in. Users can still override with new Options().block(false). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 62 metrics, 9 unstable metrics. Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.61.0-SNAPSHOT~f1647d5d5e, baseline=1.61.0-SNAPSHOT~52d1e1902e
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.059 s) : 0, 1058873
Total [baseline] (11.096 s) : 0, 11096432
Agent [candidate] (1.056 s) : 0, 1056174
Total [candidate] (11.075 s) : 0, 11074794
section appsec
Agent [baseline] (1.246 s) : 0, 1245569
Total [baseline] (11.132 s) : 0, 11132324
Agent [candidate] (1.251 s) : 0, 1250569
Total [candidate] (11.083 s) : 0, 11083474
section iast
Agent [baseline] (1.232 s) : 0, 1232194
Total [baseline] (11.324 s) : 0, 11324333
Agent [candidate] (1.245 s) : 0, 1245192
Total [candidate] (11.357 s) : 0, 11356921
section profiling
Agent [baseline] (1.186 s) : 0, 1185593
Total [baseline] (11.045 s) : 0, 11045071
Agent [candidate] (1.18 s) : 0, 1180418
Total [candidate] (10.99 s) : 0, 10989629
gantt
title petclinic - break down per module: candidate=1.61.0-SNAPSHOT~f1647d5d5e, baseline=1.61.0-SNAPSHOT~52d1e1902e
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.203 ms) : 0, 1203
crashtracking [candidate] (1.186 ms) : 0, 1186
BytebuddyAgent [baseline] (628.611 ms) : 0, 628611
BytebuddyAgent [candidate] (626.212 ms) : 0, 626212
AgentMeter [baseline] (29.075 ms) : 0, 29075
AgentMeter [candidate] (29.002 ms) : 0, 29002
GlobalTracer [baseline] (257.31 ms) : 0, 257310
GlobalTracer [candidate] (256.211 ms) : 0, 256211
AppSec [baseline] (31.499 ms) : 0, 31499
AppSec [candidate] (31.363 ms) : 0, 31363
Debugger [baseline] (59.367 ms) : 0, 59367
Debugger [candidate] (59.268 ms) : 0, 59268
Remote Config [baseline] (619.241 µs) : 0, 619
Remote Config [candidate] (612.901 µs) : 0, 613
Telemetry [baseline] (8.601 ms) : 0, 8601
Telemetry [candidate] (8.593 ms) : 0, 8593
Flare Poller [baseline] (6.559 ms) : 0, 6559
Flare Poller [candidate] (7.891 ms) : 0, 7891
section appsec
crashtracking [baseline] (1.195 ms) : 0, 1195
crashtracking [candidate] (1.206 ms) : 0, 1206
BytebuddyAgent [baseline] (657.874 ms) : 0, 657874
BytebuddyAgent [candidate] (660.768 ms) : 0, 660768
AgentMeter [baseline] (12.063 ms) : 0, 12063
AgentMeter [candidate] (12.079 ms) : 0, 12079
GlobalTracer [baseline] (257.839 ms) : 0, 257839
GlobalTracer [candidate] (258.999 ms) : 0, 258999
IAST [baseline] (23.95 ms) : 0, 23950
IAST [candidate] (24.186 ms) : 0, 24186
AppSec [baseline] (177.32 ms) : 0, 177320
AppSec [candidate] (177.992 ms) : 0, 177992
Debugger [baseline] (65.975 ms) : 0, 65975
Debugger [candidate] (65.923 ms) : 0, 65923
Remote Config [baseline] (570.003 µs) : 0, 570
Remote Config [candidate] (569.149 µs) : 0, 569
Telemetry [baseline] (8.952 ms) : 0, 8952
Telemetry [candidate] (9.003 ms) : 0, 9003
Flare Poller [baseline] (3.627 ms) : 0, 3627
Flare Poller [candidate] (3.614 ms) : 0, 3614
section iast
crashtracking [baseline] (1.199 ms) : 0, 1199
crashtracking [candidate] (1.225 ms) : 0, 1225
BytebuddyAgent [baseline] (800.974 ms) : 0, 800974
BytebuddyAgent [candidate] (808.64 ms) : 0, 808640
AgentMeter [baseline] (11.592 ms) : 0, 11592
AgentMeter [candidate] (11.933 ms) : 0, 11933
GlobalTracer [baseline] (247.339 ms) : 0, 247339
GlobalTracer [candidate] (250.269 ms) : 0, 250269
IAST [baseline] (25.025 ms) : 0, 25025
IAST [candidate] (25.612 ms) : 0, 25612
AppSec [baseline] (26.344 ms) : 0, 26344
AppSec [candidate] (26.911 ms) : 0, 26911
Debugger [baseline] (63.607 ms) : 0, 63607
Debugger [candidate] (64.653 ms) : 0, 64653
Remote Config [baseline] (500.634 µs) : 0, 501
Remote Config [candidate] (522.072 µs) : 0, 522
Telemetry [baseline] (14.633 ms) : 0, 14633
Telemetry [candidate] (14.279 ms) : 0, 14279
Flare Poller [baseline] (4.875 ms) : 0, 4875
Flare Poller [candidate] (4.806 ms) : 0, 4806
section profiling
crashtracking [baseline] (1.183 ms) : 0, 1183
crashtracking [candidate] (1.175 ms) : 0, 1175
BytebuddyAgent [baseline] (684.964 ms) : 0, 684964
BytebuddyAgent [candidate] (681.266 ms) : 0, 681266
AgentMeter [baseline] (8.644 ms) : 0, 8644
AgentMeter [candidate] (8.566 ms) : 0, 8566
GlobalTracer [baseline] (215.795 ms) : 0, 215795
GlobalTracer [candidate] (214.968 ms) : 0, 214968
AppSec [baseline] (31.988 ms) : 0, 31988
AppSec [candidate] (31.893 ms) : 0, 31893
Debugger [baseline] (62.336 ms) : 0, 62336
Debugger [candidate] (63.549 ms) : 0, 63549
Remote Config [baseline] (584.363 µs) : 0, 584
Remote Config [candidate] (593.489 µs) : 0, 593
Telemetry [baseline] (10.551 ms) : 0, 10551
Telemetry [candidate] (9.035 ms) : 0, 9035
Flare Poller [baseline] (4.346 ms) : 0, 4346
Flare Poller [candidate] (4.23 ms) : 0, 4230
ProfilingAgent [baseline] (94.145 ms) : 0, 94145
ProfilingAgent [candidate] (94.508 ms) : 0, 94508
Profiling [baseline] (94.716 ms) : 0, 94716
Profiling [candidate] (95.079 ms) : 0, 95079
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.61.0-SNAPSHOT~f1647d5d5e, baseline=1.61.0-SNAPSHOT~52d1e1902e
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.071 s) : 0, 1071081
Total [baseline] (8.878 s) : 0, 8878482
Agent [candidate] (1.057 s) : 0, 1057070
Total [candidate] (8.858 s) : 0, 8857559
section iast
Agent [baseline] (1.227 s) : 0, 1226681
Total [baseline] (9.572 s) : 0, 9571546
Agent [candidate] (1.227 s) : 0, 1226549
Total [candidate] (9.576 s) : 0, 9575510
gantt
title insecure-bank - break down per module: candidate=1.61.0-SNAPSHOT~f1647d5d5e, baseline=1.61.0-SNAPSHOT~52d1e1902e
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.247 ms) : 0, 1247
crashtracking [candidate] (1.197 ms) : 0, 1197
BytebuddyAgent [baseline] (636.533 ms) : 0, 636533
BytebuddyAgent [candidate] (628.659 ms) : 0, 628659
AgentMeter [baseline] (29.136 ms) : 0, 29136
AgentMeter [candidate] (28.942 ms) : 0, 28942
GlobalTracer [baseline] (258.554 ms) : 0, 258554
GlobalTracer [candidate] (257.117 ms) : 0, 257117
AppSec [baseline] (31.78 ms) : 0, 31780
AppSec [candidate] (31.461 ms) : 0, 31461
Debugger [baseline] (59.344 ms) : 0, 59344
Debugger [candidate] (58.795 ms) : 0, 58795
Remote Config [baseline] (620.972 µs) : 0, 621
Remote Config [candidate] (619.292 µs) : 0, 619
Telemetry [baseline] (8.743 ms) : 0, 8743
Telemetry [candidate] (8.652 ms) : 0, 8652
Flare Poller [baseline] (8.789 ms) : 0, 8789
Flare Poller [candidate] (5.659 ms) : 0, 5659
section iast
crashtracking [baseline] (1.203 ms) : 0, 1203
crashtracking [candidate] (1.212 ms) : 0, 1212
BytebuddyAgent [baseline] (796.389 ms) : 0, 796389
BytebuddyAgent [candidate] (795.128 ms) : 0, 795128
AgentMeter [baseline] (11.348 ms) : 0, 11348
AgentMeter [candidate] (11.331 ms) : 0, 11331
GlobalTracer [baseline] (247.215 ms) : 0, 247215
GlobalTracer [candidate] (247.548 ms) : 0, 247548
IAST [baseline] (25.128 ms) : 0, 25128
IAST [candidate] (25.212 ms) : 0, 25212
AppSec [baseline] (28.084 ms) : 0, 28084
AppSec [candidate] (26.491 ms) : 0, 26491
Debugger [baseline] (60.947 ms) : 0, 60947
Debugger [candidate] (63.335 ms) : 0, 63335
Remote Config [baseline] (516.828 µs) : 0, 517
Remote Config [candidate] (521.306 µs) : 0, 521
Telemetry [baseline] (14.861 ms) : 0, 14861
Telemetry [candidate] (14.851 ms) : 0, 14851
Flare Poller [baseline] (4.884 ms) : 0, 4884
Flare Poller [candidate] (4.91 ms) : 0, 4910
LoadParameters
See matching parameters
SummaryFound 1 performance improvements and 1 performance regressions! Performance is the same for 18 metrics, 16 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~f1647d5d5e, baseline=1.61.0-SNAPSHOT~52d1e1902e
dateFormat X
axisFormat %s
section baseline
no_agent (1.181 ms) : 1169, 1192
. : milestone, 1181,
iast (3.22 ms) : 3173, 3267
. : milestone, 3220,
iast_FULL (5.845 ms) : 5786, 5903
. : milestone, 5845,
iast_GLOBAL (3.522 ms) : 3468, 3575
. : milestone, 3522,
profiling (2.017 ms) : 2000, 2034
. : milestone, 2017,
tracing (1.789 ms) : 1772, 1805
. : milestone, 1789,
section candidate
no_agent (1.185 ms) : 1173, 1196
. : milestone, 1185,
iast (3.231 ms) : 3186, 3277
. : milestone, 3231,
iast_FULL (5.755 ms) : 5698, 5811
. : milestone, 5755,
iast_GLOBAL (3.444 ms) : 3392, 3496
. : milestone, 3444,
profiling (1.974 ms) : 1957, 1990
. : milestone, 1974,
tracing (1.788 ms) : 1772, 1803
. : milestone, 1788,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~f1647d5d5e, baseline=1.61.0-SNAPSHOT~52d1e1902e
dateFormat X
axisFormat %s
section baseline
no_agent (16.916 ms) : 16745, 17086
. : milestone, 16916,
appsec (18.352 ms) : 18169, 18536
. : milestone, 18352,
code_origins (17.713 ms) : 17534, 17893
. : milestone, 17713,
iast (18.103 ms) : 17920, 18286
. : milestone, 18103,
profiling (18.687 ms) : 18502, 18872
. : milestone, 18687,
tracing (17.786 ms) : 17608, 17965
. : milestone, 17786,
section candidate
no_agent (18.992 ms) : 18793, 19190
. : milestone, 18992,
appsec (18.428 ms) : 18238, 18619
. : milestone, 18428,
code_origins (17.946 ms) : 17766, 18127
. : milestone, 17946,
iast (17.603 ms) : 17427, 17780
. : milestone, 17603,
profiling (18.622 ms) : 18437, 18807
. : milestone, 18622,
tracing (18.104 ms) : 17924, 18285
. : milestone, 18104,
DacapoParameters
See matching parameters
SummaryFound 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 0 unstable metrics.
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~f1647d5d5e, baseline=1.61.0-SNAPSHOT~52d1e1902e
dateFormat X
axisFormat %s
section baseline
no_agent (14.998 s) : 14998000, 14998000
. : milestone, 14998000,
appsec (15.041 s) : 15041000, 15041000
. : milestone, 15041000,
iast (18.359 s) : 18359000, 18359000
. : milestone, 18359000,
iast_GLOBAL (17.728 s) : 17728000, 17728000
. : milestone, 17728000,
profiling (15.227 s) : 15227000, 15227000
. : milestone, 15227000,
tracing (15.297 s) : 15297000, 15297000
. : milestone, 15297000,
section candidate
no_agent (15.301 s) : 15301000, 15301000
. : milestone, 15301000,
appsec (14.898 s) : 14898000, 14898000
. : milestone, 14898000,
iast (17.798 s) : 17798000, 17798000
. : milestone, 17798000,
iast_GLOBAL (17.694 s) : 17694000, 17694000
. : milestone, 17694000,
profiling (15.567 s) : 15567000, 15567000
. : milestone, 15567000,
tracing (15.289 s) : 15289000, 15289000
. : milestone, 15289000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~f1647d5d5e, baseline=1.61.0-SNAPSHOT~52d1e1902e
dateFormat X
axisFormat %s
section baseline
no_agent (1.473 ms) : 1461, 1484
. : milestone, 1473,
appsec (3.789 ms) : 3569, 4009
. : milestone, 3789,
iast (2.252 ms) : 2183, 2321
. : milestone, 2252,
iast_GLOBAL (2.3 ms) : 2231, 2370
. : milestone, 2300,
profiling (2.08 ms) : 2025, 2135
. : milestone, 2080,
tracing (2.062 ms) : 2009, 2116
. : milestone, 2062,
section candidate
no_agent (1.468 ms) : 1457, 1480
. : milestone, 1468,
appsec (2.525 ms) : 2470, 2580
. : milestone, 2525,
iast (2.247 ms) : 2179, 2316
. : milestone, 2247,
iast_GLOBAL (2.297 ms) : 2227, 2366
. : milestone, 2297,
profiling (2.107 ms) : 2050, 2163
. : milestone, 2107,
tracing (2.061 ms) : 2008, 2115
. : milestone, 2061,
|
Member
Author
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in Use ⏳ Processing |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Does This Do
Change Options.block default from false to true so that evaluate() follows the remote is_blocking_enabled setting without requiring explicit opt-in. Users can still override with new Options().block(false).
Motivation
Additional Notes
Contributor Checklist
type:and (comp:orinst:) labels in addition to any other useful labelsclose,fix, or any linking keywords when referencing an issueUse
solvesinstead, and assign the PR milestone to the issueJira ticket: APPSEC-61436
Note: Once your PR is ready to merge, add it to the merge queue by commenting
/merge./merge -ccancels the queue request./merge -f --reason "reason"skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.