AAP service activation origin metric #8934
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sezen-datadog
force-pushed
the
sezen.leblay/APPSEC-57893-appsec-enabled-metric
branch
from
437276f to
03e4246
Compare
BenchmarksStartupParameters
See matching parameters
SummaryFound 7 performance improvements and 1 performance regressions! Performance is the same for 48 metrics, 15 unstable metrics.
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~d05b090830, baseline=1.50.0-SNAPSHOT~2442b1fa6d
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.029 s) : 0, 1029099
Total [baseline] (8.522 s) : 0, 8522366
Agent [candidate] (999.419 ms) : 0, 999419
Total [candidate] (8.542 s) : 0, 8541979
section iast
Agent [baseline] (1.168 s) : 0, 1168038
Total [baseline] (9.214 s) : 0, 9213909
Agent [candidate] (1.127 s) : 0, 1126673
Total [candidate] (9.204 s) : 0, 9203564
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.15 s) : 0, 1149864
Total [baseline] (9.16 s) : 0, 9159535
Agent [candidate] (1.127 s) : 0, 1127145
Total [candidate] (9.177 s) : 0, 9177115
section iast_TELEMETRY_OFF
Agent [baseline] (1.144 s) : 0, 1144306
Total [baseline] (9.159 s) : 0, 9159243
Agent [candidate] (1.139 s) : 0, 1139060
Total [candidate] (9.267 s) : 0, 9266562
gantt
title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~d05b090830, baseline=1.50.0-SNAPSHOT~2442b1fa6d
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (688.694 ms) : 0, 688694
BytebuddyAgent [candidate] (688.74 ms) : 0, 688740
GlobalTracer [baseline] (242.087 ms) : 0, 242087
GlobalTracer [candidate] (242.125 ms) : 0, 242125
AppSec [baseline] (56.206 ms) : 0, 56206
AppSec [candidate] (29.632 ms) : 0, 29632
Debugger [baseline] (6.24 ms) : 0, 6240
Debugger [candidate] (6.113 ms) : 0, 6113
Remote Config [baseline] (727.914 µs) : 0, 728
Remote Config [candidate] (654.86 µs) : 0, 655
Telemetry [baseline] (11.424 ms) : 0, 11424
Telemetry [candidate] (8.347 ms) : 0, 8347
section iast
BytebuddyAgent [baseline] (815.478 ms) : 0, 815478
BytebuddyAgent [candidate] (802.983 ms) : 0, 802983
GlobalTracer [baseline] (233.901 ms) : 0, 233901
GlobalTracer [candidate] (231.065 ms) : 0, 231065
IAST [baseline] (28.256 ms) : 0, 28256
IAST [candidate] (26.17 ms) : 0, 26170
AppSec [baseline] (51.973 ms) : 0, 51973
AppSec [candidate] (28.594 ms) : 0, 28594
Debugger [baseline] (6.017 ms) : 0, 6017
Debugger [candidate] (5.761 ms) : 0, 5761
Remote Config [baseline] (614.804 µs) : 0, 615
Remote Config [candidate] (599.161 µs) : 0, 599
Telemetry [baseline] (7.996 ms) : 0, 7996
Telemetry [candidate] (7.956 ms) : 0, 7956
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (801.315 ms) : 0, 801315
BytebuddyAgent [candidate] (803.281 ms) : 0, 803281
GlobalTracer [baseline] (230.615 ms) : 0, 230615
GlobalTracer [candidate] (231.163 ms) : 0, 231163
IAST [baseline] (24.715 ms) : 0, 24715
IAST [candidate] (28.613 ms) : 0, 28613
AppSec [baseline] (55.13 ms) : 0, 55130
AppSec [candidate] (26.368 ms) : 0, 26368
Debugger [baseline] (5.979 ms) : 0, 5979
Debugger [candidate] (5.822 ms) : 0, 5822
Remote Config [baseline] (606.29 µs) : 0, 606
Remote Config [candidate] (582.831 µs) : 0, 583
Telemetry [baseline] (7.952 ms) : 0, 7952
Telemetry [candidate] (7.817 ms) : 0, 7817
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (797.35 ms) : 0, 797350
BytebuddyAgent [candidate] (812.405 ms) : 0, 812405
GlobalTracer [baseline] (229.715 ms) : 0, 229715
GlobalTracer [candidate] (233.875 ms) : 0, 233875
IAST [baseline] (24.993 ms) : 0, 24993
IAST [candidate] (29.627 ms) : 0, 29627
AppSec [baseline] (54.37 ms) : 0, 54370
AppSec [candidate] (25.187 ms) : 0, 25187
Debugger [baseline] (5.981 ms) : 0, 5981
Debugger [candidate] (5.884 ms) : 0, 5884
Remote Config [baseline] (609.057 µs) : 0, 609
Remote Config [candidate] (603.056 µs) : 0, 603
Telemetry [baseline] (7.81 ms) : 0, 7810
Telemetry [candidate] (7.753 ms) : 0, 7753
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~d05b090830, baseline=1.50.0-SNAPSHOT~2442b1fa6d
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.027 s) : 0, 1026535
Total [baseline] (11.164 s) : 0, 11163613
Agent [candidate] (993.889 ms) : 0, 993889
Total [candidate] (11.135 s) : 0, 11135442
section appsec
Agent [baseline] (1.174 s) : 0, 1174044
Total [baseline] (11.176 s) : 0, 11176477
Agent [candidate] (1.161 s) : 0, 1161446
Total [candidate] (11.302 s) : 0, 11301767
section iast
Agent [baseline] (1.152 s) : 0, 1151877
Total [baseline] (11.363 s) : 0, 11362904
Agent [candidate] (1.127 s) : 0, 1126974
Total [candidate] (11.366 s) : 0, 11365507
section profiling
Agent [baseline] (1.266 s) : 0, 1266203
Total [baseline] (11.393 s) : 0, 11393479
Agent [candidate] (1.249 s) : 0, 1249490
Total [candidate] (11.577 s) : 0, 11577398
gantt
title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~d05b090830, baseline=1.50.0-SNAPSHOT~2442b1fa6d
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (685.106 ms) : 0, 685106
BytebuddyAgent [candidate] (685.064 ms) : 0, 685064
GlobalTracer [baseline] (240.822 ms) : 0, 240822
GlobalTracer [candidate] (240.566 ms) : 0, 240566
AppSec [baseline] (57.22 ms) : 0, 57220
AppSec [candidate] (29.488 ms) : 0, 29488
Debugger [baseline] (6.14 ms) : 0, 6140
Debugger [candidate] (6.082 ms) : 0, 6082
Remote Config [baseline] (730.852 µs) : 0, 731
Remote Config [candidate] (656.217 µs) : 0, 656
Telemetry [baseline] (12.864 ms) : 0, 12864
Telemetry [candidate] (8.249 ms) : 0, 8249
section appsec
BytebuddyAgent [baseline] (704.638 ms) : 0, 704638
BytebuddyAgent [candidate] (702.564 ms) : 0, 702564
GlobalTracer [baseline] (235.339 ms) : 0, 235339
GlobalTracer [candidate] (233.831 ms) : 0, 233831
IAST [baseline] (21.857 ms) : 0, 21857
IAST [candidate] (21.823 ms) : 0, 21823
AppSec [baseline] (174.956 ms) : 0, 174956
AppSec [candidate] (164.992 ms) : 0, 164992
Debugger [baseline] (5.931 ms) : 0, 5931
Debugger [candidate] (5.896 ms) : 0, 5896
Remote Config [baseline] (624.67 µs) : 0, 625
Remote Config [candidate] (679.031 µs) : 0, 679
Telemetry [baseline] (7.235 ms) : 0, 7235
Telemetry [candidate] (8.2 ms) : 0, 8200
section iast
BytebuddyAgent [baseline] (802.586 ms) : 0, 802586
BytebuddyAgent [candidate] (803.053 ms) : 0, 803053
GlobalTracer [baseline] (231.451 ms) : 0, 231451
GlobalTracer [candidate] (230.925 ms) : 0, 230925
IAST [baseline] (27.938 ms) : 0, 27938
IAST [candidate] (28.509 ms) : 0, 28509
AppSec [baseline] (51.726 ms) : 0, 51726
AppSec [candidate] (26.802 ms) : 0, 26802
Debugger [baseline] (6.034 ms) : 0, 6034
Debugger [candidate] (5.843 ms) : 0, 5843
Remote Config [baseline] (607.816 µs) : 0, 608
Remote Config [candidate] (580.146 µs) : 0, 580
Telemetry [baseline] (7.992 ms) : 0, 7992
Telemetry [candidate] (7.759 ms) : 0, 7759
section profiling
BytebuddyAgent [baseline] (675.956 ms) : 0, 675956
BytebuddyAgent [candidate] (680.679 ms) : 0, 680679
GlobalTracer [baseline] (360.429 ms) : 0, 360429
GlobalTracer [candidate] (362.925 ms) : 0, 362925
AppSec [baseline] (61.179 ms) : 0, 61179
AppSec [candidate] (32.071 ms) : 0, 32071
Debugger [baseline] (6.036 ms) : 0, 6036
Debugger [candidate] (10.577 ms) : 0, 10577
Remote Config [baseline] (659.962 µs) : 0, 660
Remote Config [candidate] (706.629 µs) : 0, 707
Telemetry [baseline] (8.125 ms) : 0, 8125
Telemetry [candidate] (9.067 ms) : 0, 9067
ProfilingAgent [baseline] (103.109 ms) : 0, 103109
ProfilingAgent [candidate] (102.381 ms) : 0, 102381
Profiling [baseline] (103.133 ms) : 0, 103133
Profiling [candidate] (102.406 ms) : 0, 102406
LoadParameters
See matching parameters
SummaryFound 3 performance improvements and 2 performance regressions! Performance is the same for 6 metrics, 17 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~d05b090830, baseline=1.50.0-SNAPSHOT~2442b1fa6d
dateFormat X
axisFormat %s
section baseline
no_agent (8.809 ms) : 8783, 8834
. : milestone, 8809,
appsec (9.483 ms) : 9435, 9531
. : milestone, 9483,
code_origins (203.207 µs) : 176, 231
. : milestone, 203,
iast (900.687 µs) : 850, 951
. : milestone, 901,
profiling (8.049 ms) : 8020, 8078
. : milestone, 8049,
tracing (0.151 µs) : 0, 0
. : milestone, 0,
section candidate
no_agent (7.69 ms) : 7668, 7711
. : milestone, 7690,
appsec (4.036 ms) : 3726, 4346
. : milestone, 4036,
code_origins (1.326 ms) : 1051, 1601
. : milestone, 1326,
iast (2.699 ms) : 2400, 2999
. : milestone, 2699,
profiling (10.408 ms) : 10340, 10477
. : milestone, 10408,
tracing (8.16 ms) : 8134, 8186
. : milestone, 8160,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~d05b090830, baseline=1.50.0-SNAPSHOT~2442b1fa6d
dateFormat X
axisFormat %s
section baseline
no_agent (384.797 µs) : 365, 404
. : milestone, 385,
iast (534.695 µs) : 511, 558
. : milestone, 535,
iast_FULL (747.473 µs) : 724, 771
. : milestone, 747,
iast_GLOBAL (583.547 µs) : 560, 607
. : milestone, 584,
iast_HARDCODED_SECRET_DISABLED (533.246 µs) : 512, 555
. : milestone, 533,
iast_INACTIVE (482.363 µs) : 459, 506
. : milestone, 482,
iast_TELEMETRY_OFF (514.781 µs) : 493, 537
. : milestone, 515,
tracing (483.232 µs) : 461, 506
. : milestone, 483,
section candidate
no_agent (386.584 µs) : 366, 407
. : milestone, 387,
iast (529.752 µs) : 506, 553
. : milestone, 530,
iast_FULL (744.959 µs) : 721, 768
. : milestone, 745,
iast_GLOBAL (578.84 µs) : 554, 604
. : milestone, 579,
iast_HARDCODED_SECRET_DISABLED (528.108 µs) : 505, 551
. : milestone, 528,
iast_INACTIVE (476.979 µs) : 454, 500
. : milestone, 477,
iast_TELEMETRY_OFF (519.116 µs) : 496, 542
. : milestone, 519,
tracing (465.752 µs) : 444, 488
. : milestone, 466,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics. Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~d05b090830, baseline=1.50.0-SNAPSHOT~2442b1fa6d
dateFormat X
axisFormat %s
section baseline
no_agent (15.111 s) : 15111000, 15111000
. : milestone, 15111000,
appsec (14.701 s) : 14701000, 14701000
. : milestone, 14701000,
iast (18.722 s) : 18722000, 18722000
. : milestone, 18722000,
iast_GLOBAL (18.062 s) : 18062000, 18062000
. : milestone, 18062000,
profiling (15.22 s) : 15220000, 15220000
. : milestone, 15220000,
tracing (14.891 s) : 14891000, 14891000
. : milestone, 14891000,
section candidate
no_agent (15.245 s) : 15245000, 15245000
. : milestone, 15245000,
appsec (15.049 s) : 15049000, 15049000
. : milestone, 15049000,
iast (18.842 s) : 18842000, 18842000
. : milestone, 18842000,
iast_GLOBAL (18.013 s) : 18013000, 18013000
. : milestone, 18013000,
profiling (15.986 s) : 15986000, 15986000
. : milestone, 15986000,
tracing (14.859 s) : 14859000, 14859000
. : milestone, 14859000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~d05b090830, baseline=1.50.0-SNAPSHOT~2442b1fa6d
dateFormat X
axisFormat %s
section baseline
no_agent (1.477 ms) : 1465, 1488
. : milestone, 1477,
appsec (2.409 ms) : 2360, 2458
. : milestone, 2409,
iast (2.189 ms) : 2128, 2250
. : milestone, 2189,
iast_GLOBAL (2.241 ms) : 2179, 2303
. : milestone, 2241,
profiling (2.059 ms) : 2008, 2109
. : milestone, 2059,
tracing (2.014 ms) : 1966, 2062
. : milestone, 2014,
section candidate
no_agent (1.48 ms) : 1468, 1491
. : milestone, 1480,
appsec (2.405 ms) : 2356, 2454
. : milestone, 2405,
iast (2.2 ms) : 2139, 2262
. : milestone, 2200,
iast_GLOBAL (2.234 ms) : 2172, 2296
. : milestone, 2234,
profiling (2.05 ms) : 2000, 2100
. : milestone, 2050,
tracing (2.002 ms) : 1955, 2050
. : milestone, 2002,
|
sezen-datadog
force-pushed
the
sezen.leblay/APPSEC-57893-appsec-enabled-metric
branch
8 times, most recently
from
20f69bd to
7b3cf77
Compare
sezen-datadog
requested review from
smola,
manuel-alvarez-alvarez and
dougqh
and removed request for
a team
|
Hi! 👋 Thanks for your pull request! 🎉 To help us review it, please make sure to:
If you need help, please check our contributing guidelines. |
| import java.util.concurrent.ArrayBlockingQueue; | ||
| import java.util.concurrent.BlockingQueue; | ||
|
|
||
| public class AppSecMetricCollector implements MetricCollector<MetricCollector.Metric> { |
There was a problem hiding this comment.
Traditionally we've used the WafMetricCollector as the source for all appsec metrics, although I'm not against about this new one.
| // Periodically report AppSec enabled status | ||
| final ProductActivation appSecActivation = Config.get().getAppSecActivation(); | ||
| if (appSecActivation == ProductActivation.FULLY_ENABLED | ||
| || ConfigOrigin.REMOTE == appsecOrigin) { // remote enablement can happen later |
There was a problem hiding this comment.
If I understood correctly, you have to report the metric in every heartbeat when appsec is fully enabled (not only when it's configured via RC),
There was a problem hiding this comment.
yes, but we have to report remote enablement as well
| @@ -521,6 +523,11 @@ private void setAppSecActivation(final AppSecFeatures.Asm asm) { | |||
| newState = tracerConfig.getAppSecActivation() == ProductActivation.FULLY_ENABLED; | |||
There was a problem hiding this comment.
If we receive a null via RC then we should go back to report the metric with the locally configured origin (default, env_var, ...)
| if (newState) { | ||
| AppSecMetricCollector.appsecOrigin = ConfigOrigin.REMOTE; | ||
| } else { | ||
| AppSecMetricCollector.appsecOrigin = null; |
There was a problem hiding this comment.
In here the source is still RC, the difference is that appsec is disabled so we should not report the gauge
sezen-datadog
changed the base branch from
master
to
sezen.leblay/upgrade-libddwaf-java-1.23
sezen-datadog
force-pushed
the
sezen.leblay/upgrade-libddwaf-java-1.23
branch
from
6c47b1e to
42f13c9
Compare
sezen-datadog
force-pushed
the
sezen.leblay/APPSEC-57893-appsec-enabled-metric
branch
3 times, most recently
from
fafba5f to
daf546f
Compare
sezen-datadog
force-pushed
the
sezen.leblay/upgrade-libddwaf-java-1.23
branch
from
42f13c9 to
5b79c2f
Compare
sezen-datadog
force-pushed
the
sezen.leblay/upgrade-libddwaf-java-1.23
branch
from
5b79c2f to
8212f64
Compare
Signed-off-by: sezen.leblay <[email protected]>
sezen-datadog
force-pushed
the
sezen.leblay/APPSEC-57893-appsec-enabled-metric
branch
from
957025f to
af32b13
Compare
…lay/APPSEC-57893-appsec-enabled-metric
|
possibly canceled @robertpi tell us if opened or closed with regards to what is decided on the origin metric |
sezen-datadog
force-pushed
the
sezen.leblay/upgrade-libddwaf-java-1.23
branch
from
c837ca4 to
bad8ea7
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What Does This Do
It sends appsec.enabled metric every 60 seconds (existing heartbeat value for telemetry)
Motivation
Additional Notes
Contributor Checklist
type:and (comp:orinst:) labels in addition to any usefull labelsclose,fixor any linking keywords when referencing an issue.Use
solvesinstead, and assign the PR milestone to the issueJira ticket: APPSEC-57893