Use OpenLineage root parent information to generate trace id #8726

mobuchowski · 2025-04-23T12:13:13Z

What Does This Do

This utilizes newly added root parent information in OpenLineage context to deterministically generate TraceId and SpanId out of it.

Motivation

This allows us to generate consistent traces across data jobs.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: DJM-744

pr-commenter · 2025-04-23T12:31:22Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mobuchowski/openlineage-trace-context
git_commit_date	1745405337	1745448578
git_commit_sha	`22535c7`	`dd7895c`
release_version	1.49.0-SNAPSHOT~22535c74a4	1.49.0-SNAPSHOT~dd7895c99a

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1745451615	1745451615
ci_job_id	909937866	909937866
ci_pipeline_id	63140467	63140467
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-ekfrqbyz-project-304-concurrent-1-v9jqbyrz 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-ekfrqbyz-project-304-concurrent-1-v9jqbyrz 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 57 metrics, 14 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.49.0-SNAPSHOT~dd7895c99a, baseline=1.49.0-SNAPSHOT~22535c74a4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.068 s) : 0, 1067740
Total [baseline] (8.737 s) : 0, 8736866
Agent [candidate] (1.067 s) : 0, 1066688
Total [candidate] (8.751 s) : 0, 8751380
section iast
Agent [baseline] (1.188 s) : 0, 1188073
Total [baseline] (9.266 s) : 0, 9265505
Agent [candidate] (1.196 s) : 0, 1196084
Total [candidate] (9.246 s) : 0, 9246332
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.194 s) : 0, 1194244
Total [baseline] (9.242 s) : 0, 9242377
Agent [candidate] (1.197 s) : 0, 1197374
Total [candidate] (9.298 s) : 0, 9297911
section iast_TELEMETRY_OFF
Agent [baseline] (1.184 s) : 0, 1183651
Total [baseline] (9.356 s) : 0, 9355973
Agent [candidate] (1.192 s) : 0, 1192023
Total [candidate] (9.293 s) : 0, 9293304

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.068 s	-
Agent	iast	1.188 s	120.333 ms (11.3%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.194 s	126.505 ms (11.8%)
Agent	iast_TELEMETRY_OFF	1.184 s	115.912 ms (10.9%)
Total	tracing	8.737 s	-
Total	iast	9.266 s	528.639 ms (6.1%)
Total	iast_HARDCODED_SECRET_DISABLED	9.242 s	505.511 ms (5.8%)
Total	iast_TELEMETRY_OFF	9.356 s	619.107 ms (7.1%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.067 s	-
Agent	iast	1.196 s	129.395 ms (12.1%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.197 s	130.686 ms (12.3%)
Agent	iast_TELEMETRY_OFF	1.192 s	125.334 ms (11.7%)
Total	tracing	8.751 s	-
Total	iast	9.246 s	494.953 ms (5.7%)
Total	iast_HARDCODED_SECRET_DISABLED	9.298 s	546.531 ms (6.2%)
Total	iast_TELEMETRY_OFF	9.293 s	541.924 ms (6.2%)

gantt
    title insecure-bank - break down per module: candidate=1.49.0-SNAPSHOT~dd7895c99a, baseline=1.49.0-SNAPSHOT~22535c74a4

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (729.832 ms) : 0, 729832
BytebuddyAgent [candidate] (728.611 ms) : 0, 728611
GlobalTracer [baseline] (241.465 ms) : 0, 241465
GlobalTracer [candidate] (241.5 ms) : 0, 241500
AppSec [baseline] (55.645 ms) : 0, 55645
AppSec [candidate] (55.037 ms) : 0, 55037
Debugger [baseline] (6.509 ms) : 0, 6509
Debugger [candidate] (8.064 ms) : 0, 8064
Remote Config [baseline] (694.156 µs) : 0, 694
Remote Config [candidate] (690.731 µs) : 0, 691
Telemetry [baseline] (9.927 ms) : 0, 9927
Telemetry [candidate] (9.212 ms) : 0, 9212
section iast
BytebuddyAgent [baseline] (841.998 ms) : 0, 841998
BytebuddyAgent [candidate] (848.537 ms) : 0, 848537
GlobalTracer [baseline] (230.08 ms) : 0, 230080
GlobalTracer [candidate] (230.935 ms) : 0, 230935
IAST [baseline] (23.633 ms) : 0, 23633
IAST [candidate] (22.977 ms) : 0, 22977
AppSec [baseline] (54.651 ms) : 0, 54651
AppSec [candidate] (55.022 ms) : 0, 55022
Debugger [baseline] (4.239 ms) : 0, 4239
Debugger [candidate] (4.223 ms) : 0, 4223
Remote Config [baseline] (617.136 µs) : 0, 617
Remote Config [candidate] (606.872 µs) : 0, 607
Telemetry [baseline] (8.723 ms) : 0, 8723
Telemetry [candidate] (8.754 ms) : 0, 8754
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (846.229 ms) : 0, 846229
BytebuddyAgent [candidate] (848.833 ms) : 0, 848833
GlobalTracer [baseline] (231.585 ms) : 0, 231585
GlobalTracer [candidate] (231.573 ms) : 0, 231573
IAST [baseline] (24.122 ms) : 0, 24122
IAST [candidate] (24.118 ms) : 0, 24118
AppSec [baseline] (51.366 ms) : 0, 51366
AppSec [candidate] (54.724 ms) : 0, 54724
Debugger [baseline] (4.248 ms) : 0, 4248
Debugger [candidate] (4.309 ms) : 0, 4309
Remote Config [baseline] (607.539 µs) : 0, 608
Remote Config [candidate] (616.699 µs) : 0, 617
Telemetry [baseline] (8.764 ms) : 0, 8764
Telemetry [candidate] (8.816 ms) : 0, 8816
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (838.323 ms) : 0, 838323
BytebuddyAgent [candidate] (845.339 ms) : 0, 845339
GlobalTracer [baseline] (229.77 ms) : 0, 229770
GlobalTracer [candidate] (230.902 ms) : 0, 230902
IAST [baseline] (22.412 ms) : 0, 22412
IAST [candidate] (23.352 ms) : 0, 23352
AppSec [baseline] (56.327 ms) : 0, 56327
AppSec [candidate] (55.432 ms) : 0, 55432
Debugger [baseline] (4.195 ms) : 0, 4195
Debugger [candidate] (4.177 ms) : 0, 4177
Remote Config [baseline] (621.829 µs) : 0, 622
Remote Config [candidate] (621.752 µs) : 0, 622
Telemetry [baseline] (8.628 ms) : 0, 8628
Telemetry [candidate] (8.717 ms) : 0, 8717

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.49.0-SNAPSHOT~dd7895c99a, baseline=1.49.0-SNAPSHOT~22535c74a4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.059 s) : 0, 1058777
Total [baseline] (10.465 s) : 0, 10465487
Agent [candidate] (1.06 s) : 0, 1060236
Total [candidate] (10.492 s) : 0, 10492157
section appsec
Agent [baseline] (1.202 s) : 0, 1201697
Total [baseline] (10.785 s) : 0, 10784934
Agent [candidate] (1.2 s) : 0, 1200119
Total [candidate] (10.84 s) : 0, 10840324
section iast
Agent [baseline] (1.19 s) : 0, 1190251
Total [baseline] (11.094 s) : 0, 11094030
Agent [candidate] (1.19 s) : 0, 1190364
Total [candidate] (11.069 s) : 0, 11069500
section profiling
Agent [baseline] (1.312 s) : 0, 1311550
Total [baseline] (10.907 s) : 0, 10907430
Agent [candidate] (1.33 s) : 0, 1330119
Total [candidate] (10.984 s) : 0, 10983800

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.059 s	-
Agent	appsec	1.202 s	142.919 ms (13.5%)
Agent	iast	1.19 s	131.474 ms (12.4%)
Agent	profiling	1.312 s	252.773 ms (23.9%)
Total	tracing	10.465 s	-
Total	appsec	10.785 s	319.446 ms (3.1%)
Total	iast	11.094 s	628.542 ms (6.0%)
Total	profiling	10.907 s	441.943 ms (4.2%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.06 s	-
Agent	appsec	1.2 s	139.883 ms (13.2%)
Agent	iast	1.19 s	130.129 ms (12.3%)
Agent	profiling	1.33 s	269.883 ms (25.5%)
Total	tracing	10.492 s	-
Total	appsec	10.84 s	348.168 ms (3.3%)
Total	iast	11.069 s	577.343 ms (5.5%)
Total	profiling	10.984 s	491.643 ms (4.7%)

gantt
    title petclinic - break down per module: candidate=1.49.0-SNAPSHOT~dd7895c99a, baseline=1.49.0-SNAPSHOT~22535c74a4

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (723.877 ms) : 0, 723877
BytebuddyAgent [candidate] (724.414 ms) : 0, 724414
GlobalTracer [baseline] (239.557 ms) : 0, 239557
GlobalTracer [candidate] (239.746 ms) : 0, 239746
AppSec [baseline] (54.823 ms) : 0, 54823
AppSec [candidate] (54.599 ms) : 0, 54599
Debugger [baseline] (5.806 ms) : 0, 5806
Debugger [candidate] (6.612 ms) : 0, 6612
Remote Config [baseline] (705.321 µs) : 0, 705
Remote Config [candidate] (706.329 µs) : 0, 706
Telemetry [baseline] (10.63 ms) : 0, 10630
Telemetry [candidate] (10.666 ms) : 0, 10666
section appsec
BytebuddyAgent [baseline] (742.287 ms) : 0, 742287
BytebuddyAgent [candidate] (740.825 ms) : 0, 740825
GlobalTracer [baseline] (236.348 ms) : 0, 236348
GlobalTracer [candidate] (235.91 ms) : 0, 235910
IAST [baseline] (21.825 ms) : 0, 21825
IAST [candidate] (21.814 ms) : 0, 21814
AppSec [baseline] (175.172 ms) : 0, 175172
AppSec [candidate] (175.869 ms) : 0, 175869
Debugger [baseline] (4.608 ms) : 0, 4608
Debugger [candidate] (4.218 ms) : 0, 4218
Remote Config [baseline] (630.474 µs) : 0, 630
Remote Config [candidate] (633.688 µs) : 0, 634
Telemetry [baseline] (8.181 ms) : 0, 8181
Telemetry [candidate] (8.193 ms) : 0, 8193
section iast
BytebuddyAgent [baseline] (842.874 ms) : 0, 842874
BytebuddyAgent [candidate] (843.038 ms) : 0, 843038
GlobalTracer [baseline] (230.417 ms) : 0, 230417
GlobalTracer [candidate] (230.7 ms) : 0, 230700
IAST [baseline] (23.755 ms) : 0, 23755
IAST [candidate] (22.971 ms) : 0, 22971
AppSec [baseline] (56.068 ms) : 0, 56068
AppSec [candidate] (51.958 ms) : 0, 51958
Debugger [baseline] (4.27 ms) : 0, 4270
Debugger [candidate] (4.234 ms) : 0, 4234
Remote Config [baseline] (612.842 µs) : 0, 613
Remote Config [candidate] (602.747 µs) : 0, 603
Telemetry [baseline] (8.875 ms) : 0, 8875
Telemetry [candidate] (8.766 ms) : 0, 8766
section profiling
BytebuddyAgent [baseline] (714.814 ms) : 0, 714814
BytebuddyAgent [candidate] (725.295 ms) : 0, 725295
GlobalTracer [baseline] (376.749 ms) : 0, 376749
GlobalTracer [candidate] (380.965 ms) : 0, 380965
AppSec [baseline] (54.278 ms) : 0, 54278
AppSec [candidate] (54.448 ms) : 0, 54448
Debugger [baseline] (4.432 ms) : 0, 4432
Debugger [candidate] (4.464 ms) : 0, 4464
Remote Config [baseline] (658.281 µs) : 0, 658
Remote Config [candidate] (668.595 µs) : 0, 669
Telemetry [baseline] (8.988 ms) : 0, 8988
Telemetry [candidate] (9.084 ms) : 0, 9084
ProfilingAgent [baseline] (101.246 ms) : 0, 101246
ProfilingAgent [candidate] (104.182 ms) : 0, 104182
Profiling [baseline] (101.271 ms) : 0, 101271
Profiling [candidate] (104.208 ms) : 0, 104208

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-04-23T23:10:16	2025-04-23T23:18:02
git_branch	master	mobuchowski/openlineage-trace-context
git_commit_date	1745405337	1745448578
git_commit_sha	`22535c7`	`dd7895c`
release_version	1.49.0-SNAPSHOT~22535c74a4	1.49.0-SNAPSHOT~dd7895c99a
start_time	2025-04-23T23:10:02	2025-04-23T23:17:48

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1745450679	1745450679
ci_job_id	909937869	909937869
ci_pipeline_id	63140467	63140467
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-9smzurdp-project-304-concurrent-0-saww673o 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-9smzurdp-project-304-concurrent-0-saww673o 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 17 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:petclinic:profiling	better [-106.572µs; -53.588µs] or [-6.779%; -3.409%]	unstable [-454.550op/s; +682.470op/s] or [-15.341%; +23.033%]	1.492ms	3076.923op/s	1.572ms	2962.963op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~dd7895c99a, baseline=1.49.0-SNAPSHOT~22535c74a4
    dateFormat X
    axisFormat %s
section baseline
no_agent (383.434 µs) : 363, 404
.   : milestone, 383,
iast (510.167 µs) : 488, 532
.   : milestone, 510,
iast_FULL (727.233 µs) : 705, 749
.   : milestone, 727,
iast_GLOBAL (561.624 µs) : 540, 583
.   : milestone, 562,
iast_HARDCODED_SECRET_DISABLED (511.376 µs) : 490, 533
.   : milestone, 511,
iast_INACTIVE (455.689 µs) : 435, 477
.   : milestone, 456,
iast_TELEMETRY_OFF (499.471 µs) : 478, 521
.   : milestone, 499,
tracing (454.704 µs) : 434, 475
.   : milestone, 455,
section candidate
no_agent (381.093 µs) : 362, 401
.   : milestone, 381,
iast (513.033 µs) : 491, 535
.   : milestone, 513,
iast_FULL (732.409 µs) : 710, 754
.   : milestone, 732,
iast_GLOBAL (557.079 µs) : 535, 579
.   : milestone, 557,
iast_HARDCODED_SECRET_DISABLED (507.384 µs) : 486, 529
.   : milestone, 507,
iast_INACTIVE (465.787 µs) : 443, 488
.   : milestone, 466,
iast_TELEMETRY_OFF (501.02 µs) : 479, 523
.   : milestone, 501,
tracing (458.835 µs) : 438, 480
.   : milestone, 459,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	383.434 µs [363.155 µs, 403.713 µs]	-
iast	510.167 µs [488.172 µs, 532.162 µs]	126.733 µs (33.1%)
iast_FULL	727.233 µs [705.145 µs, 749.32 µs]	343.799 µs (89.7%)
iast_GLOBAL	561.624 µs [539.793 µs, 583.455 µs]	178.19 µs (46.5%)
iast_HARDCODED_SECRET_DISABLED	511.376 µs [489.567 µs, 533.186 µs]	127.942 µs (33.4%)
iast_INACTIVE	455.689 µs [434.508 µs, 476.87 µs]	72.255 µs (18.8%)
iast_TELEMETRY_OFF	499.471 µs [477.728 µs, 521.215 µs]	116.037 µs (30.3%)
tracing	454.704 µs [434.078 µs, 475.329 µs]	71.27 µs (18.6%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	381.093 µs [361.505 µs, 400.68 µs]	-
iast	513.033 µs [491.198 µs, 534.869 µs]	131.941 µs (34.6%)
iast_FULL	732.409 µs [710.323 µs, 754.494 µs]	351.316 µs (92.2%)
iast_GLOBAL	557.079 µs [535.472 µs, 578.685 µs]	175.986 µs (46.2%)
iast_HARDCODED_SECRET_DISABLED	507.384 µs [485.83 µs, 528.937 µs]	126.291 µs (33.1%)
iast_INACTIVE	465.787 µs [443.241 µs, 488.332 µs]	84.694 µs (22.2%)
iast_TELEMETRY_OFF	501.02 µs [479.082 µs, 522.958 µs]	119.928 µs (31.5%)
tracing	458.835 µs [438.091 µs, 479.579 µs]	77.742 µs (20.4%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~dd7895c99a, baseline=1.49.0-SNAPSHOT~22535c74a4
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.353 ms) : 1333, 1372
.   : milestone, 1353,
appsec (1.725 ms) : 1701, 1749
.   : milestone, 1725,
appsec_no_iast (1.732 ms) : 1709, 1755
.   : milestone, 1732,
code_origins (1.693 ms) : 1666, 1720
.   : milestone, 1693,
iast (1.515 ms) : 1491, 1538
.   : milestone, 1515,
profiling (1.572 ms) : 1547, 1597
.   : milestone, 1572,
tracing (1.519 ms) : 1495, 1543
.   : milestone, 1519,
section candidate
no_agent (1.363 ms) : 1343, 1383
.   : milestone, 1363,
appsec (1.74 ms) : 1716, 1764
.   : milestone, 1740,
appsec_no_iast (1.735 ms) : 1711, 1758
.   : milestone, 1735,
code_origins (1.691 ms) : 1664, 1717
.   : milestone, 1691,
iast (1.528 ms) : 1504, 1552
.   : milestone, 1528,
profiling (1.492 ms) : 1468, 1516
.   : milestone, 1492,
tracing (1.507 ms) : 1483, 1530
.   : milestone, 1507,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.353 ms [1.333 ms, 1.372 ms]	-
appsec	1.725 ms [1.701 ms, 1.749 ms]	372.307 µs (27.5%)
appsec_no_iast	1.732 ms [1.709 ms, 1.755 ms]	379.357 µs (28.0%)
code_origins	1.693 ms [1.666 ms, 1.72 ms]	340.228 µs (25.2%)
iast	1.515 ms [1.491 ms, 1.538 ms]	162.031 µs (12.0%)
profiling	1.572 ms [1.547 ms, 1.597 ms]	219.339 µs (16.2%)
tracing	1.519 ms [1.495 ms, 1.543 ms]	166.554 µs (12.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.363 ms [1.343 ms, 1.383 ms]	-
appsec	1.74 ms [1.716 ms, 1.764 ms]	377.007 µs (27.7%)
appsec_no_iast	1.735 ms [1.711 ms, 1.758 ms]	371.826 µs (27.3%)
code_origins	1.691 ms [1.664 ms, 1.717 ms]	327.572 µs (24.0%)
iast	1.528 ms [1.504 ms, 1.552 ms]	164.99 µs (12.1%)
profiling	1.492 ms [1.468 ms, 1.516 ms]	128.98 µs (9.5%)
tracing	1.507 ms [1.483 ms, 1.53 ms]	143.539 µs (10.5%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mobuchowski/openlineage-trace-context
git_commit_date	1745405337	1745448578
git_commit_sha	`22535c7`	`dd7895c`
release_version	1.49.0-SNAPSHOT~22535c74a4	1.49.0-SNAPSHOT~dd7895c99a

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1745451187	1745451187
ci_job_id	909937870	909937870
ci_pipeline_id	63140467	63140467
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-4i48fg5e-project-304-concurrent-1-upewejg9 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-4i48fg5e-project-304-concurrent-1-upewejg9 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~dd7895c99a, baseline=1.49.0-SNAPSHOT~22535c74a4
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.911 s) : 14911000, 14911000
.   : milestone, 14911000,
appsec (15.254 s) : 15254000, 15254000
.   : milestone, 15254000,
iast (18.848 s) : 18848000, 18848000
.   : milestone, 18848000,
iast_GLOBAL (18.209 s) : 18209000, 18209000
.   : milestone, 18209000,
profiling (15.475 s) : 15475000, 15475000
.   : milestone, 15475000,
tracing (15.228 s) : 15228000, 15228000
.   : milestone, 15228000,
section candidate
no_agent (14.907 s) : 14907000, 14907000
.   : milestone, 14907000,
appsec (15.001 s) : 15001000, 15001000
.   : milestone, 15001000,
iast (18.664 s) : 18664000, 18664000
.   : milestone, 18664000,
iast_GLOBAL (18.095 s) : 18095000, 18095000
.   : milestone, 18095000,
profiling (15.737 s) : 15737000, 15737000
.   : milestone, 15737000,
tracing (14.931 s) : 14931000, 14931000
.   : milestone, 14931000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.911 s [14.911 s, 14.911 s]	-
appsec	15.254 s [15.254 s, 15.254 s]	343.0 ms (2.3%)
iast	18.848 s [18.848 s, 18.848 s]	3.937 s (26.4%)
iast_GLOBAL	18.209 s [18.209 s, 18.209 s]	3.298 s (22.1%)
profiling	15.475 s [15.475 s, 15.475 s]	564.0 ms (3.8%)
tracing	15.228 s [15.228 s, 15.228 s]	317.0 ms (2.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.907 s [14.907 s, 14.907 s]	-
appsec	15.001 s [15.001 s, 15.001 s]	94.0 ms (0.6%)
iast	18.664 s [18.664 s, 18.664 s]	3.757 s (25.2%)
iast_GLOBAL	18.095 s [18.095 s, 18.095 s]	3.188 s (21.4%)
profiling	15.737 s [15.737 s, 15.737 s]	830.0 ms (5.6%)
tracing	14.931 s [14.931 s, 14.931 s]	24.0 ms (0.2%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~dd7895c99a, baseline=1.49.0-SNAPSHOT~22535c74a4
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.473 ms) : 1462, 1485
.   : milestone, 1473,
appsec (2.347 ms) : 2303, 2391
.   : milestone, 2347,
iast (2.117 ms) : 2062, 2173
.   : milestone, 2117,
iast_GLOBAL (2.17 ms) : 2113, 2226
.   : milestone, 2170,
profiling (2.451 ms) : 2269, 2633
.   : milestone, 2451,
tracing (1.946 ms) : 1904, 1989
.   : milestone, 1946,
section candidate
no_agent (1.472 ms) : 1461, 1484
.   : milestone, 1472,
appsec (2.346 ms) : 2303, 2390
.   : milestone, 2346,
iast (2.115 ms) : 2060, 2171
.   : milestone, 2115,
iast_GLOBAL (2.162 ms) : 2107, 2218
.   : milestone, 2162,
profiling (1.989 ms) : 1944, 2034
.   : milestone, 1989,
tracing (1.947 ms) : 1904, 1989
.   : milestone, 1947,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.473 ms [1.462 ms, 1.485 ms]	-
appsec	2.347 ms [2.303 ms, 2.391 ms]	873.941 µs (59.3%)
iast	2.117 ms [2.062 ms, 2.173 ms]	644.083 µs (43.7%)
iast_GLOBAL	2.17 ms [2.113 ms, 2.226 ms]	696.324 µs (47.3%)
profiling	2.451 ms [2.269 ms, 2.633 ms]	977.404 µs (66.3%)
tracing	1.946 ms [1.904 ms, 1.989 ms]	473.118 µs (32.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.472 ms [1.461 ms, 1.484 ms]	-
appsec	2.346 ms [2.303 ms, 2.39 ms]	874.005 µs (59.4%)
iast	2.115 ms [2.06 ms, 2.171 ms]	642.951 µs (43.7%)
iast_GLOBAL	2.162 ms [2.107 ms, 2.218 ms]	689.902 µs (46.9%)
profiling	1.989 ms [1.944 ms, 2.034 ms]	516.336 µs (35.1%)
tracing	1.947 ms [1.904 ms, 1.989 ms]	474.421 µs (32.2%)

...on/spark/src/main/java/datadog/trace/instrumentation/spark/AbstractDatadogSparkListener.java

...tation/spark/src/main/java/datadog/trace/instrumentation/spark/OpenlineageParentContext.java

… of root parent information Signed-off-by: Maciej Obuchowski <[email protected]>

Signed-off-by: Maciej Obuchowski <[email protected]>

* spark instrumentation use openlineage trace id if can generate it out of root parent information Signed-off-by: Maciej Obuchowski <[email protected]> * spark: use OpenLineage context to generate trace/span id if present Signed-off-by: Maciej Obuchowski <[email protected]> * code review fixes, add version tag Signed-off-by: Maciej Obuchowski <[email protected]> --------- Signed-off-by: Maciej Obuchowski <[email protected]>

…8726)" This reverts commit 44de9a3.

| Package | Type | Package file | Manager | Update | Change | |---|---|---|---|---|---| | [com.google.cloud:google-cloud-logging](https://github.com/googleapis/java-logging) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `3.22.2` -> `3.22.3` | | [com.google.cloud:google-cloud-datastore](https://github.com/googleapis/java-datastore) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.28.0` -> `2.28.1` | | [com.datadoghq:dd-trace-api](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.48.2` -> `1.49.0` | | [software.amazon.awssdk:sdk-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.36` -> `2.31.37` | | [software.amazon.awssdk:sqs](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.36` -> `2.31.37` | | [software.amazon.awssdk:dynamodb-enhanced](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.36` -> `2.31.37` | | [software.amazon.awssdk:dynamodb](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.36` -> `2.31.37` | | [software.amazon.awssdk:aws-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.36` -> `2.31.37` | | [software.amazon.awssdk:bom](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.36` -> `2.31.37` | | [software.amazon.awssdk:auth](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.36` -> `2.31.37` | --- ### Release Notes <details> <summary>googleapis/java-logging (com.google.cloud:google-cloud-logging)</summary> ### [`v3.22.3`](https://github.com/googleapis/java-logging/blob/HEAD/CHANGELOG.md#3223-2025-05-06) ##### Bug Fixes - **deps:** Update the Java code generator (gapic-generator-java) to 2.56.3 ([844f4fa](googleapis/java-logging@844f4fa)) ##### Dependencies - Update dependency com.google.cloud:sdk-platform-java-config to v3.46.3 ([#1801](googleapis/java-logging#1801)) ([d7aa7bc](googleapis/java-logging@d7aa7bc)) - Update dependency com.google.cloud:sdk-platform-java-config to v3.47.0 ([#1803](googleapis/java-logging#1803)) ([5967ffe](googleapis/java-logging@5967ffe)) - Update googleapis/sdk-platform-java action to v2.57.0 ([#1804](googleapis/java-logging#1804)) ([e9a27ec](googleapis/java-logging@e9a27ec)) </details> <details> <summary>googleapis/java-datastore (com.google.cloud:google-cloud-datastore)</summary> ### [`v2.28.1`](https://github.com/googleapis/java-datastore/blob/HEAD/CHANGELOG.md#2281-2025-05-06) ##### Dependencies - Update dependency com.google.cloud:sdk-platform-java-config to v3.47.0 ([#1841](googleapis/java-datastore#1841)) ([ac393e6](googleapis/java-datastore@ac393e6)) - Update googleapis/sdk-platform-java action to v2.57.0 ([#1842](googleapis/java-datastore#1842)) ([0745906](googleapis/java-datastore@0745906)) </details> <details> <summary>datadog/dd-trace-java (com.datadoghq:dd-trace-api)</summary> ### [`v1.49.0`](https://github.com/DataDog/dd-trace-java/releases/tag/v1.49.0): 1.49.0 ### Components #### Configuration at Runtime - ✨ Add process tags as list to remote config payload ([#8705](DataDog/dd-trace-java#8705) - [@amarziali](https://github.com/amarziali)) #### Continuous Integration Visibility - 🐛 Add span propagation for Pekko scheduled tasks ([#8765](DataDog/dd-trace-java#8765) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Update test.retry_reason to use full name of the feature ([#8689](DataDog/dd-trace-java#8689) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - 🧹 Remove unused TestEventsHandler methods ([#8674](DataDog/dd-trace-java#8674) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) #### Dynamic Instrumentation - 🐛 Fix exclude identifiers normalization ([#8742](DataDog/dd-trace-java#8742) - [@jpbempel](https://github.com/jpbempel)) - ✨ Make source file tracking asynchronous ([#8684](DataDog/dd-trace-java#8684) - [@jpbempel](https://github.com/jpbempel)) - ✨ Add scope filtering for symbol extraction ([#8676](DataDog/dd-trace-java#8676) - [@jpbempel](https://github.com/jpbempel)) - ✨ Add support for [@key](https://github.com/key) and [@value](https://github.com/value) for Map filtering ([#8669](DataDog/dd-trace-java#8669) - [@jpbempel](https://github.com/jpbempel)) #### Library Injection - ✨ Add system property to force injection of the tracing library even though multiple javaagents have been detected ([#8697](DataDog/dd-trace-java#8697) - [@cecile75](https://github.com/cecile75)) #### Metrics - ✨ Allow dogstatsd port to be configurable with DD_DOGSTATSD_PORT ([#8693](DataDog/dd-trace-java#8693) - [@randomanderson](https://github.com/randomanderson)) #### Profiling - ✨ Bump ddprof-java to 1.25.1 ([#8750](DataDog/dd-trace-java#8750) - [@jbachorik](https://github.com/jbachorik)) - 🐛 Remove cleanup-on-shutdown for temporary files ([#8746](DataDog/dd-trace-java#8746) - [@jbachorik](https://github.com/jbachorik)) - ✨⚡ Replace a regex-based SMAP parser with a hand-crafted one ([#8730](DataDog/dd-trace-java#8730) - [@jbachorik](https://github.com/jbachorik)) - ✨ Improve error reporting on profiler startup ([#8714](DataDog/dd-trace-java#8714) - [@jbachorik](https://github.com/jbachorik)) - ✨ Exclude ProxyLeakTask exception from exception profiling ([#8666](DataDog/dd-trace-java#8666) - [@jbachorik](https://github.com/jbachorik)) - ✨ Use jvmstat for JDKs 9+ programmatically ([#8641](DataDog/dd-trace-java#8641) - [@MattAlp](https://github.com/MattAlp)) #### Telemetry - ✨ Allow dogstatsd port to be configurable with DD_DOGSTATSD_PORT ([#8693](DataDog/dd-trace-java#8693) - [@randomanderson](https://github.com/randomanderson)) - 🐛 Fix appsec.waf.requests telemetry metric ([#8644](DataDog/dd-trace-java#8644) - [@jandro996](https://github.com/jandro996)) #### Tracer core - ✨ Exclude jackson afterburner dynamic classes from instrumentation ([#8747](DataDog/dd-trace-java#8747) - [@amarziali](https://github.com/amarziali)) - ✨ Introduce Java 8 bytecode bridge for instrumentation API ([#8736](DataDog/dd-trace-java#8736) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ⚡🧹 Use byte-buddy classes optimized for Java8+ ([#8735](DataDog/dd-trace-java#8735) - [@mcculls](https://github.com/mcculls)) - 🐛 Do not set the hibernate or datanucleus span service name when disabled ([#8727](DataDog/dd-trace-java#8727) - [@ygree](https://github.com/ygree)) - ✨ Update bytebuddy and ASM to support JDK 24 ([#8720](DataDog/dd-trace-java#8720) - [@sarahchen6](https://github.com/sarahchen6)) - 🐛 Turn off JDK socket support by default ([#8715](DataDog/dd-trace-java#8715) - [@mcculls](https://github.com/mcculls)) - 🐛 Log warning when trace buffer overflow occurs ([#8712](DataDog/dd-trace-java#8712) - [@ygree](https://github.com/ygree)) - ✨🧪 Introducing an internal integration name ([#8708](DataDog/dd-trace-java#8708) - [@amarziali](https://github.com/amarziali)) - ✨ Add process tags to client stats payload ([#8704](DataDog/dd-trace-java#8704) - [@amarziali](https://github.com/amarziali)) - ✨ Collect process tags for tracing ([#8698](DataDog/dd-trace-java#8698) - [@amarziali](https://github.com/amarziali)) - ✨ Stable Config file: target system properties in process_arguments and support template variables in YamlParser ([#8690](DataDog/dd-trace-java#8690) - [@mtoffl01](https://github.com/mtoffl01)) - ✨⚡ Use prefix trie for proxy ignores ([#8678](DataDog/dd-trace-java#8678) - [@amarziali](https://github.com/amarziali)) - ✨ Allow agent to be automatically injected when running aside Log4J patch agent ([#8648](DataDog/dd-trace-java#8648) - [@paullegranddc](https://github.com/paullegranddc)) - ✨ Use jvmstat for JDKs 9+ programmatically ([#8641](DataDog/dd-trace-java#8641) - [@MattAlp](https://github.com/MattAlp)) #### Tracer internal logging - 🐛 Delete print line ([#8686](DataDog/dd-trace-java#8686) - [@sarahchen6](https://github.com/sarahchen6)) ### Instrumentations #### Akka instrumentation - 🐛 Handle reentrant scope cleanup in Akka/Pekko actor instrumentations ([#8722](DataDog/dd-trace-java#8722) - [@mcculls](https://github.com/mcculls)) #### Apache Spark instrumentation - ✨ Use OpenLineage root parent information to generate trace id ([#8726](DataDog/dd-trace-java#8726) - [@mobuchowski](https://github.com/mobuchowski)) - ✨ Spark job cancellation no longer marks application as failed ([#8701](DataDog/dd-trace-java#8701) - [@paul-laffon-dd](https://github.com/paul-laffon-dd)) #### JDBC instrumentation - 💡 Add support for sybase tds jdbc driver ([#8764](DataDog/dd-trace-java#8764) - [@amarziali](https://github.com/amarziali)) #### Kotlin instrumentation - 🐛 Take defensive copy of parent scope stack when closing nested coroutines ([#8749](DataDog/dd-trace-java#8749) - [@mcculls](https://github.com/mcculls)) #### Reactor instrumentation - ✨⚡ Do not inspect reactor context when not needed ([#8745](DataDog/dd-trace-java#8745) - [@amarziali](https://github.com/amarziali)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6pm every weekday,before 2am every weekday" in timezone Australia/Melbourne, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Never, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). GitOrigin-RevId: 795f347ae34d056efc1194c2f606cee7bca1beea

mobuchowski added type: enhancement inst: apache spark Apache Spark instrumentation labels Apr 23, 2025

mobuchowski requested a review from a team as a code owner April 23, 2025 12:13

aboitreaud reviewed Apr 23, 2025

View reviewed changes

...on/spark/src/main/java/datadog/trace/instrumentation/spark/AbstractDatadogSparkListener.java Outdated Show resolved Hide resolved

PerfectSlayer reviewed Apr 23, 2025

View reviewed changes

mobuchowski added 2 commits April 23, 2025 17:54

spark instrumentation use openlineage trace id if can generate it out…

42fc779

… of root parent information Signed-off-by: Maciej Obuchowski <[email protected]>

spark: use OpenLineage context to generate trace/span id if present

cc7cdde

Signed-off-by: Maciej Obuchowski <[email protected]>

mobuchowski force-pushed the mobuchowski/openlineage-trace-context branch from df87a93 to 4df065e Compare April 23, 2025 15:55

kr-igor approved these changes Apr 23, 2025

View reviewed changes

mobuchowski force-pushed the mobuchowski/openlineage-trace-context branch 2 times, most recently from 6b462d1 to cc72f34 Compare April 23, 2025 22:26

code review fixes, add version tag

dd7895c

Signed-off-by: Maciej Obuchowski <[email protected]>

mobuchowski force-pushed the mobuchowski/openlineage-trace-context branch from cc72f34 to dd7895c Compare April 23, 2025 22:49

mobuchowski merged commit 44de9a3 into master Apr 24, 2025
459 checks passed

mobuchowski deleted the mobuchowski/openlineage-trace-context branch April 24, 2025 11:57

github-actions bot added this to the 1.49.0 milestone Apr 24, 2025

PerfectSlayer added a commit that referenced this pull request Apr 25, 2025

Revert "Use OpenLineage root parent information to generate trace id (#…

5baa904

…8726)" This reverts commit 44de9a3.

PerfectSlayer mentioned this pull request Apr 25, 2025

Revert "Use OpenLineage root parent information to generate trace id" #8737

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Use OpenLineage root parent information to generate trace id #8726

Use OpenLineage root parent information to generate trace id #8726

Uh oh!

mobuchowski commented Apr 23, 2025 •

edited by jira bot

Loading

Uh oh!

pr-commenter bot commented Apr 23, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Use OpenLineage root parent information to generate trace id #8726

Use OpenLineage root parent information to generate trace id #8726

Uh oh!

Conversation

mobuchowski commented Apr 23, 2025 • edited by jira bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Contributor Checklist

Uh oh!

pr-commenter bot commented Apr 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mobuchowski commented Apr 23, 2025 •

edited by jira bot

Loading

pr-commenter bot commented Apr 23, 2025 •

edited

Loading