Llmobs java sdk integration #8966
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…g provided in the override
* add APIs for llm obs * add llm message class to support llm spans * follow java convention of naming Id instead of ID * add codeowners
* add APIs for llm obs * add llm message class to support llm spans * follow java convention of naming Id instead of ID * add codeowners
* add APIs for llm obs * add llm message class to support llm spans * add llm message class to support llm spans * impl llmobs agent and llmobs apis * support llm messages with tool calls * handle default model name and provider * rm unneeded file * spotless * add APIs for llm obs sdk (#8135) * add APIs for llm obs * add llm message class to support llm spans * follow java convention of naming Id instead of ID * add codeowners * rename ID to Id according to java naming conventions * Undo change to integrations-core submodule * fix build gradle * rm empty line * fix test
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 12 unstable metrics. Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~8659686578, baseline=1.50.0-SNAPSHOT~089cf8fd47
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.024 s) : 0, 1023996
Total [baseline] (11.174 s) : 0, 11173840
Agent [candidate] (1.029 s) : 0, 1028861
Total [candidate] (11.061 s) : 0, 11061183
section appsec
Agent [baseline] (1.165 s) : 0, 1164856
Total [baseline] (11.227 s) : 0, 11226705
Agent [candidate] (1.162 s) : 0, 1162244
Total [candidate] (11.177 s) : 0, 11176827
section iast
Agent [baseline] (1.15 s) : 0, 1149767
Total [baseline] (11.314 s) : 0, 11313947
Agent [candidate] (1.157 s) : 0, 1157420
Total [candidate] (11.364 s) : 0, 11363782
section profiling
Agent [baseline] (1.264 s) : 0, 1263884
Total [baseline] (11.408 s) : 0, 11408030
Agent [candidate] (1.274 s) : 0, 1273931
Total [candidate] (11.452 s) : 0, 11451862
gantt
title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~8659686578, baseline=1.50.0-SNAPSHOT~089cf8fd47
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (684.202 ms) : 0, 684202
BytebuddyAgent [candidate] (688.011 ms) : 0, 688011
GlobalTracer [baseline] (240.808 ms) : 0, 240808
GlobalTracer [candidate] (241.666 ms) : 0, 241666
AppSec [baseline] (57.007 ms) : 0, 57007
AppSec [candidate] (56.453 ms) : 0, 56453
Debugger [baseline] (6.191 ms) : 0, 6191
Debugger [candidate] (6.141 ms) : 0, 6141
Remote Config [baseline] (730.381 µs) : 0, 730
Remote Config [candidate] (734.104 µs) : 0, 734
Telemetry [baseline] (11.387 ms) : 0, 11387
Telemetry [candidate] (12.125 ms) : 0, 12125
section appsec
BytebuddyAgent [baseline] (701.607 ms) : 0, 701607
BytebuddyAgent [candidate] (700.094 ms) : 0, 700094
GlobalTracer [baseline] (238.171 ms) : 0, 238171
GlobalTracer [candidate] (237.716 ms) : 0, 237716
AppSec [baseline] (176.467 ms) : 0, 176467
AppSec [candidate] (176.092 ms) : 0, 176092
Debugger [baseline] (5.987 ms) : 0, 5987
Debugger [candidate] (5.958 ms) : 0, 5958
Remote Config [baseline] (635.006 µs) : 0, 635
Remote Config [candidate] (628.183 µs) : 0, 628
Telemetry [baseline] (7.392 ms) : 0, 7392
Telemetry [candidate] (7.32 ms) : 0, 7320
IAST [baseline] (22.05 ms) : 0, 22050
IAST [candidate] (21.794 ms) : 0, 21794
section iast
BytebuddyAgent [baseline] (801.675 ms) : 0, 801675
BytebuddyAgent [candidate] (806.659 ms) : 0, 806659
GlobalTracer [baseline] (230.516 ms) : 0, 230516
GlobalTracer [candidate] (231.318 ms) : 0, 231318
AppSec [baseline] (51.107 ms) : 0, 51107
AppSec [candidate] (52.697 ms) : 0, 52697
Debugger [baseline] (6.038 ms) : 0, 6038
Debugger [candidate] (6.088 ms) : 0, 6088
Remote Config [baseline] (597.004 µs) : 0, 597
Remote Config [candidate] (611.322 µs) : 0, 611
Telemetry [baseline] (7.926 ms) : 0, 7926
Telemetry [candidate] (8.03 ms) : 0, 8030
IAST [baseline] (28.449 ms) : 0, 28449
IAST [candidate] (28.185 ms) : 0, 28185
section profiling
BytebuddyAgent [baseline] (674.62 ms) : 0, 674620
BytebuddyAgent [candidate] (680.739 ms) : 0, 680739
GlobalTracer [baseline] (359.656 ms) : 0, 359656
GlobalTracer [candidate] (361.343 ms) : 0, 361343
AppSec [baseline] (61.363 ms) : 0, 61363
AppSec [candidate] (61.844 ms) : 0, 61844
Debugger [baseline] (6.069 ms) : 0, 6069
Debugger [candidate] (6.137 ms) : 0, 6137
Remote Config [baseline] (660.489 µs) : 0, 660
Remote Config [candidate] (664.096 µs) : 0, 664
Telemetry [baseline] (8.057 ms) : 0, 8057
Telemetry [candidate] (8.191 ms) : 0, 8191
ProfilingAgent [baseline] (102.702 ms) : 0, 102702
ProfilingAgent [candidate] (103.974 ms) : 0, 103974
Profiling [baseline] (102.726 ms) : 0, 102726
Profiling [candidate] (103.998 ms) : 0, 103998
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~8659686578, baseline=1.50.0-SNAPSHOT~089cf8fd47
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.031 s) : 0, 1030770
Total [baseline] (8.543 s) : 0, 8542628
Agent [candidate] (1.023 s) : 0, 1022796
Total [candidate] (8.493 s) : 0, 8492543
section iast
Agent [baseline] (1.15 s) : 0, 1149996
Total [baseline] (9.16 s) : 0, 9160209
Agent [candidate] (1.152 s) : 0, 1151668
Total [candidate] (9.165 s) : 0, 9165072
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.156 s) : 0, 1156342
Total [baseline] (9.142 s) : 0, 9142016
Agent [candidate] (1.154 s) : 0, 1153536
Total [candidate] (9.128 s) : 0, 9128049
section iast_TELEMETRY_OFF
Agent [baseline] (1.15 s) : 0, 1150397
Total [baseline] (9.153 s) : 0, 9153232
Agent [candidate] (1.155 s) : 0, 1155423
Total [candidate] (9.222 s) : 0, 9222070
gantt
title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~8659686578, baseline=1.50.0-SNAPSHOT~089cf8fd47
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (688.337 ms) : 0, 688337
BytebuddyAgent [candidate] (683.899 ms) : 0, 683899
GlobalTracer [baseline] (241.292 ms) : 0, 241292
GlobalTracer [candidate] (240.262 ms) : 0, 240262
AppSec [baseline] (57.778 ms) : 0, 57778
AppSec [candidate] (57.522 ms) : 0, 57522
Debugger [baseline] (6.168 ms) : 0, 6168
Debugger [candidate] (6.108 ms) : 0, 6108
Remote Config [baseline] (750.336 µs) : 0, 750
Remote Config [candidate] (737.633 µs) : 0, 738
Telemetry [baseline] (12.722 ms) : 0, 12722
Telemetry [candidate] (10.688 ms) : 0, 10688
section iast
BytebuddyAgent [baseline] (802.525 ms) : 0, 802525
BytebuddyAgent [candidate] (803.607 ms) : 0, 803607
GlobalTracer [baseline] (230.488 ms) : 0, 230488
GlobalTracer [candidate] (231.146 ms) : 0, 231146
IAST [baseline] (28.668 ms) : 0, 28668
IAST [candidate] (27.75 ms) : 0, 27750
AppSec [baseline] (50.348 ms) : 0, 50348
AppSec [candidate] (51.19 ms) : 0, 51190
Debugger [baseline] (5.955 ms) : 0, 5955
Debugger [candidate] (5.938 ms) : 0, 5938
Remote Config [baseline] (601.025 µs) : 0, 601
Remote Config [candidate] (602.618 µs) : 0, 603
Telemetry [baseline] (7.904 ms) : 0, 7904
Telemetry [candidate] (7.877 ms) : 0, 7877
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (806.507 ms) : 0, 806507
BytebuddyAgent [candidate] (804.034 ms) : 0, 804034
GlobalTracer [baseline] (231.335 ms) : 0, 231335
GlobalTracer [candidate] (230.687 ms) : 0, 230687
IAST [baseline] (26.438 ms) : 0, 26438
IAST [candidate] (26.637 ms) : 0, 26637
AppSec [baseline] (53.711 ms) : 0, 53711
AppSec [candidate] (53.984 ms) : 0, 53984
Debugger [baseline] (6.028 ms) : 0, 6028
Debugger [candidate] (5.991 ms) : 0, 5991
Remote Config [baseline] (600.208 µs) : 0, 600
Remote Config [candidate] (591.763 µs) : 0, 592
Telemetry [baseline] (8.035 ms) : 0, 8035
Telemetry [candidate] (7.981 ms) : 0, 7981
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (802.399 ms) : 0, 802399
BytebuddyAgent [candidate] (805.236 ms) : 0, 805236
GlobalTracer [baseline] (230.671 ms) : 0, 230671
GlobalTracer [candidate] (232.66 ms) : 0, 232660
IAST [baseline] (28.167 ms) : 0, 28167
IAST [candidate] (28.208 ms) : 0, 28208
AppSec [baseline] (51.082 ms) : 0, 51082
AppSec [candidate] (50.443 ms) : 0, 50443
Debugger [baseline] (5.985 ms) : 0, 5985
Debugger [candidate] (5.993 ms) : 0, 5993
Remote Config [baseline] (604.56 µs) : 0, 605
Remote Config [candidate] (599.23 µs) : 0, 599
Telemetry [baseline] (7.882 ms) : 0, 7882
Telemetry [candidate] (7.912 ms) : 0, 7912
LoadParameters
See matching parameters
SummaryFound 3 performance improvements and 5 performance regressions! Performance is the same for 7 metrics, 13 unstable metrics.
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics. Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~8659686578, baseline=1.50.0-SNAPSHOT~089cf8fd47
dateFormat X
axisFormat %s
section baseline
no_agent (14.969 s) : 14969000, 14969000
. : milestone, 14969000,
appsec (14.627 s) : 14627000, 14627000
. : milestone, 14627000,
iast (18.749 s) : 18749000, 18749000
. : milestone, 18749000,
iast_GLOBAL (17.793 s) : 17793000, 17793000
. : milestone, 17793000,
profiling (15.763 s) : 15763000, 15763000
. : milestone, 15763000,
tracing (14.896 s) : 14896000, 14896000
. : milestone, 14896000,
section candidate
no_agent (15.379 s) : 15379000, 15379000
. : milestone, 15379000,
appsec (14.834 s) : 14834000, 14834000
. : milestone, 14834000,
iast (18.525 s) : 18525000, 18525000
. : milestone, 18525000,
iast_GLOBAL (18.297 s) : 18297000, 18297000
. : milestone, 18297000,
profiling (15.617 s) : 15617000, 15617000
. : milestone, 15617000,
tracing (14.965 s) : 14965000, 14965000
. : milestone, 14965000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~8659686578, baseline=1.50.0-SNAPSHOT~089cf8fd47
dateFormat X
axisFormat %s
section baseline
no_agent (1.47 ms) : 1459, 1482
. : milestone, 1470,
appsec (2.391 ms) : 2343, 2440
. : milestone, 2391,
iast (2.175 ms) : 2114, 2236
. : milestone, 2175,
iast_GLOBAL (2.222 ms) : 2160, 2283
. : milestone, 2222,
profiling (2.032 ms) : 1982, 2082
. : milestone, 2032,
tracing (1.992 ms) : 1945, 2040
. : milestone, 1992,
section candidate
no_agent (1.47 ms) : 1459, 1481
. : milestone, 1470,
appsec (2.384 ms) : 2336, 2433
. : milestone, 2384,
iast (2.174 ms) : 2113, 2235
. : milestone, 2174,
iast_GLOBAL (2.225 ms) : 2163, 2286
. : milestone, 2225,
profiling (2.026 ms) : 1977, 2075
. : milestone, 2026,
tracing (2.001 ms) : 1954, 2049
. : milestone, 2001,
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What Does This Do
so we can get a nicer s3 link
Motivation
Additional Notes
Contributor Checklist
type:and (comp:orinst:) labels in addition to any usefull labelsclose,fixor any linking keywords when referencing an issue.Use
solvesinstead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]