Update build and test images to Alpine 3.22 #116167

richlander · 2025-05-31T01:55:47Z

Updated build and test images to Alpine 3.22.
Commented out the reference to Alpine Edge. We should only be using/running that in main. Will bring that back after the RC snap and when preparing for Alpine 3.23 becomes relevant.
A hypothesis was that Alpine Edge would not be too noisy and that we'd be clean on Alpine 3.vNext from the get-go. Let's see!

Copilot

Pull Request Overview

This PR updates the build and test images to Alpine 3.22 while commenting out the Alpine Edge builds for non‑main branches. Key changes include:

Updating image tags from alpine-3.21 to alpine-3.22 in various pipeline YAML files.
Commenting out the Alpine Edge build references to restrict their use to the main branch.
Ensuring consistency across library, installer, coreclr templates, and common pipeline resource configurations.

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File	Description
eng/pipelines/libraries/helix-queues-setup.yml	Updated Alpine image version and commented out Alpine Edge.
eng/pipelines/installer/helix-queues-setup.yml	Updated the Alpine image version for the installer pipeline.
eng/pipelines/coreclr/templates/helix-queues-setup.yml	Updated image tags for both public and internal pipelines.
eng/pipelines/common/templates/pipeline-with-resources.yml	Updated Alpine image version in common pipeline resources.

Comments suppressed due to low confidence (5)

eng/pipelines/libraries/helix-queues-setup.yml:40

Confirm that the Alpine Edge build references remain commented out for non‑main branches as intended.

# Alpine edge builds should only be used in main

eng/pipelines/libraries/helix-queues-setup.yml:44

Verify that the updated Alpine 3.22 image tag is supported by all dependent build tools and environments.

- (Alpine.322.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.22-helix-amd64

eng/pipelines/installer/helix-queues-setup.yml:36

Ensure that the update to Alpine 3.22 in the installer pipeline is validated within the AzureLinux environment.

- (Alpine.322.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.22-helix-amd64

eng/pipelines/coreclr/templates/helix-queues-setup.yml:80

Confirm that both public and internal pipeline configurations intentionally use the Alpine 3.22 image tag.

- (Alpine.322.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.22-helix-amd64

eng/pipelines/common/templates/pipeline-with-resources.yml:70

Check that the update to Alpine 3.22 in the common pipeline resource template is compatible with the overall pipeline configuration.

image: mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.22-amd64

dotnet-policy-service · 2025-05-31T01:56:30Z

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

eng/pipelines/libraries/helix-queues-setup.yml

Co-authored-by: Jan Kotas <[email protected]>

richlander · 2025-05-31T15:50:00Z

Looks like this change came out clean. Just need an approval now.

jkotas · 2025-05-31T19:09:46Z

/azp run runtime-libraries-coreclr outerloop-linux

azure-pipelines · 2025-05-31T19:09:58Z

Azure Pipelines successfully started running 1 pipeline(s).

richlander · 2025-05-31T22:12:50Z

Good call. Running outer loop occurred to me. Will do that next time.

BA is now green.

eng/pipelines/libraries/helix-queues-setup.yml

richlander · 2025-06-06T20:42:19Z

/azp run runtime-libraries-coreclr outerloop-linux

azure-pipelines · 2025-06-06T20:42:32Z

Azure Pipelines successfully started running 1 pipeline(s).

richlander · 2025-06-06T20:45:15Z

/azp run runtime-extra-platforms

azure-pipelines · 2025-06-06T20:45:30Z

Azure Pipelines successfully started running 1 pipeline(s).

richlander · 2025-06-11T03:55:50Z

/azp run runtime-libraries-coreclr outerloop-linux

azure-pipelines · 2025-06-11T03:56:03Z

Azure Pipelines successfully started running 1 pipeline(s).

richlander · 2025-06-11T03:56:13Z

/azp run runtime-extra-platforms

richlander · 2025-06-26T22:03:57Z

I removed the conditional for Alpine Edge x64. That should resolve the target queue issue. It doesn't appear to be working. Also, if it was working, it would break in the same way when we snap for RC. If it starts being noisy, we'll just remove Edge in servicing / make it the same as linux-musl-arm64. That's cheap to do.

richlander · 2025-06-27T01:30:50Z

Inner loop looks clean. I look at all the Linux failures this time.

All Linux failures are due to:

Assertion failed 'm_dfsTree->Contains(dominator) && m_dfsTree->Contains(dominated)' #117039

richlander · 2025-06-27T01:38:47Z

/azp run runtime-libraries-coreclr outerloop-linux

richlander · 2025-06-27T01:38:58Z

/azp run runtime-extra-platforms

azure-pipelines · 2025-06-27T01:38:59Z

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines · 2025-06-27T01:39:13Z

Azure Pipelines successfully started running 1 pipeline(s).

richlander · 2025-06-27T15:37:22Z

Looks pretty clean. Almost all of the Linux failures are due to the JIT issue.

There is an outerloop failure that I'm not sure about. It is seen on CentOS Stream 10 and Alpine 3.22. It would be good if someone can double-check that.

Looks like the following.

/root/helix/work/workitem/e /root/helix/work/workitem/e
  Discovering: System.Runtime.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Runtime.Tests (found 30 of 9451 test cases)
  Starting:    System.Runtime.Tests (parallel test collections = on [2 threads], stop on fail = off)
./RunTests.sh: line 178:    34 Killed                  "$RUNTIME_PATH/dotnet" exec --runtimeconfig System.Runtime.Tests.runtimeconfig.json --depsfile System.Runtime.Tests.deps.json xunit.console.dll System.Runtime.Tests.dll -xml testResults.xml -nologo -nocolor -trait category=OuterLoop -notrait category=AdditionalTimezoneChecks -notrait category=IgnoreForCI -notrait category=failing $RSP_FILE
/root/helix/work/workitem/e

richlander · 2025-06-27T16:10:19Z

It's useful to do a double-check of our configuration choices and to leave some documentation on what the expectation of this set of changes is/was. Naturally, the following "findings" are what is expected.

The PR made other changes, however, the following observations are where we made the most substantive changes that breaks the most with convention (with the intent to set new convention). These changes are intended to limit CI runs while satisfying the responsibilities that we've taken on for various users groups.

Inner loop

Note: These are the test queues run for PR runs.

For linux-x64 runtime, we see Azure Linux and Ubuntu.

  Starting Azure Pipelines Test Run coreclr linux x64 Checked no_tiered_compilation @ AzureLinux.3.Amd64.Open

For linux-x64 libraries, we see Azure Linux, CentOS Stream, and Ubuntu:

  Starting Azure Pipelines Test Run net10.0-linux-Debug-x64-coreclr_release-Ubuntu.2204.Amd64.Open
  Starting Azure Pipelines Test Run net10.0-linux-Debug-x64-coreclr_release-(AzureLinux.3.0.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:azurelinux-3.0-helix-amd64
  Starting Azure Pipelines Test Run net10.0-linux-Debug-x64-coreclr_release-(Centos.10.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:centos-stream-10-helix-amd64
  Using Queues: Ubuntu.2204.Amd64.Open+(AzureLinux.3.0.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:azurelinux-3.0-helix-amd64+(Centos.10.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:centos-stream-10-helix-amd64

For linux-musl-x64, we see latest stable Alpine:

Using Queues: (Alpine.322.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.22-helix-amd64

extra-platforms

Note: This is the same inner-loop run, for an extra set of platform, run on-demand or scheduled (not run in PR by default).

For linux-x64, we see multiple tier 2 distros and pre-release versions:

  Starting Azure Pipelines Test Run net10.0-linux-Release-x64-Libraries_Release_CoreCLR-(Debian.13.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:debian-13-helix-amd64
  Starting Azure Pipelines Test Run net10.0-linux-Release-x64-Libraries_Release_CoreCLR-(Fedora.42.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:fedora-42-helix-amd64
  Starting Azure Pipelines Test Run net10.0-linux-Release-x64-Libraries_Release_CoreCLR-(openSUSE.15.6.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:opensuse-15.6-helix-amd64
  Starting Azure Pipelines Test Run net10.0-linux-Release-x64-Libraries_Release_CoreCLR-(Ubuntu.2510.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-25.10-helix-amd64
  Using Queues: (Debian.13.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:debian-13-helix-amd64+(Fedora.42.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:fedora-42-helix-amd64+(openSUSE.15.6.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:opensuse-15.6-helix-amd64+(Ubuntu.2510.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-25.10-helix-amd64

For linux-musl-x64, we see Alpine Edge (another pre-release version):

  Sending Job to (Alpine.edge.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-edge-helix-amd64...

outerloop

Note: This is a much larger set of tests, run on-demand or scheduled (not run in PR by default).

For linux-x64, we see Azure Linux (twice; once as a VM, once in containers), Ubuntu, and CentOS Stream:

  Starting Azure Pipelines Test Run net10.0-linux-Debug-x64-CoreCLR_Debug-AzureLinux.3.Amd64.Open
  Starting Azure Pipelines Test Run net10.0-linux-Debug-x64-CoreCLR_Debug-Ubuntu.2204.Amd64.Open
  Starting Azure Pipelines Test Run net10.0-linux-Debug-x64-CoreCLR_Debug-(AzureLinux.3.0.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:azurelinux-3.0-helix-amd64
  Starting Azure Pipelines Test Run net10.0-linux-Debug-x64-CoreCLR_Debug-(Centos.10.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:centos-stream-10-helix-amd64
  Using Queues: AzureLinux.3.Amd64.Open+Ubuntu.2204.Amd64.Open+(AzureLinux.3.0.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:azurelinux-3.0-helix-amd64+(Centos.10.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:centos-stream-10-helix-amd64

For linux-musl-x64, we see latest stable Alpine:

  Starting Azure Pipelines Test Run net10.0-linux-Debug-x64-CoreCLR_Debug-(Alpine.322.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.22-helix-amd64
  Using Queues: (Alpine.322.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.22-helix-amd64

jkotas · 2025-06-27T17:52:06Z

Inner loop
Note: These are the test queues run for PR runs.
For linux-x64 runtime, we see Azure Linux and Ubuntu.

I think one queue would be enough for runtime inner loop.

The rest looks good to me.

richlander · 2025-06-27T17:54:34Z

Ok.

Current (for runtime):

    # Linux x64
    - ${{ if eq(parameters.platform, 'linux_x64') }}:
      - ${{ if eq(variables['System.TeamProject'], 'public') }}:
        - AzureLinux.3.Amd64.Open
        - (Ubuntu.2204.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-22.04-helix-amd64
      - ${{ if eq(variables['System.TeamProject'], 'internal') }}:
        - AzureLinux.3.Amd64
        - (Ubuntu.2204.Amd64)[email protected]/dotnet-buildtools/prereqs:ubuntu-22.04-helix-amd64

Proposed:

    # Linux x64
    - ${{ if eq(parameters.platform, 'linux_x64') }}:
      - ${{ if eq(variables['System.TeamProject'], 'public') }}:
        - AzureLinux.3.Amd64.Open
        - ${{ if eq(variables['System.TeamProject'], 'internal') }}:
        - AzureLinux.3.Amd64

Is that what you are expecting?

jkotas · 2025-06-27T17:55:54Z

Is that what you are expecting?

Yes, I think that's good enough for runtime inner loop.

jkotas · 2025-06-27T17:56:35Z

Almost all of the Linux failures are due to the JIT issue.

The JIT issue is fixed now. We should see much cleaner run now.

richlander · 2025-06-27T17:58:09Z

Great. I was assuming that was the reason you merged in main.

jkotas

Thank you!

richlander · 2025-06-27T18:57:19Z

/azp run runtime-libraries-coreclr outerloop-linux

richlander · 2025-06-27T18:57:30Z

/azp run runtime-extra-platforms

azure-pipelines · 2025-06-27T18:57:32Z

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines · 2025-06-27T18:57:46Z

Azure Pipelines successfully started running 1 pipeline(s).

richlander · 2025-06-27T22:05:35Z

/ba-g "Unrelated issues"

richlander · 2025-06-27T22:05:58Z

Looks like we finally this one resolved. Thanks for all the help!

richlander added 2 commits May 30, 2025 17:32

Update Alpine 3.21 -> 3.22

b366086

Update build image

92e4f92

Copilot AI review requested due to automatic review settings May 31, 2025 01:55

github-actions bot added the area-Infrastructure label May 31, 2025

github-project-automation bot added this to Runtime Infra May 31, 2025

dotnet-policy-service bot assigned richlander May 31, 2025

Copilot AI reviewed May 31, 2025

View reviewed changes

jkotas reviewed May 31, 2025

View reviewed changes

eng/pipelines/libraries/helix-queues-setup.yml Outdated Show resolved Hide resolved

Update eng/pipelines/libraries/helix-queues-setup.yml

7e0d871

Co-authored-by: Jan Kotas <[email protected]>

build-analysis bot mentioned this pull request May 31, 2025

System.Net.Quic.Tests.MsQuicTests.WriteTests failed with "System.Net.Quic.QuicException : The connection timed out from inactivity." #105177

Open

jkotas reviewed May 31, 2025

View reviewed changes

eng/pipelines/libraries/helix-queues-setup.yml Outdated Show resolved Hide resolved

richlander mentioned this pull request Jun 2, 2025

[Alpine] Add support for Alpine 3.22 dotnet/core#9865

Open

richlander added 2 commits June 6, 2025 10:53

Merge branch 'main' into pipeline-update-alpine322

78295e1

Update distribution of distros

9c25d74

This was referenced Jun 6, 2025

Test failure: baseservices/exceptions/stackoverflow/stackoverflowtester/stackoverflowtester.cmd #110173

Open

System.Net.Http.Functional.Tests timeouts #115683

Open

richlander added 2 commits June 6, 2025 16:20

Update images

894bd34

Restructure linux-x64 libraries conditions

4629f6d

Remove main conditional for linux-musl-arm64

c2c00c7

build-analysis bot mentioned this pull request Jun 27, 2025

System.Data.OleDb.Tests timeout in net48 x86 Release leg #87783

Open

Merge branch 'main' into pipeline-update-alpine322

2ee65ec

Update helix-queues-setup.yml

b6baf43

jkotas approved these changes Jun 27, 2025

View reviewed changes

richlander enabled auto-merge (squash) June 27, 2025 22:06

richlander merged commit a8c0a03 into dotnet:main Jun 27, 2025
181 of 209 checks passed

github-project-automation bot moved this to Done in Runtime Infra Jun 27, 2025

build-analysis bot mentioned this pull request Jun 27, 2025

[Android] System.IO.Pipelines.Tests.PipeWriterTests.GetMemoryFlushWithACompletedReaderNoops causes the test to fail due to memory pressure #117090

Open

richlander deleted the pipeline-update-alpine322 branch June 28, 2025 19:08

jkotas mentioned this pull request Jun 29, 2025

/azp run runtime-nativeaot-outerloop fails with "error : Helix API does not contain an entry for AzureLinux.3.ArmArch.open" #117126

Closed

Update build and test images to Alpine 3.22 #116167

Update build and test images to Alpine 3.22 #116167

Uh oh!

Conversation

richlander commented May 31, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

dotnet-policy-service bot commented May 31, 2025

Uh oh!

Uh oh!

richlander commented May 31, 2025

Uh oh!

jkotas commented May 31, 2025

Uh oh!

azure-pipelines bot commented May 31, 2025

Uh oh!

richlander commented May 31, 2025

Uh oh!

Uh oh!

richlander commented Jun 6, 2025

Uh oh!

azure-pipelines bot commented Jun 6, 2025

Uh oh!

richlander commented Jun 6, 2025

Uh oh!

azure-pipelines bot commented Jun 6, 2025

Uh oh!

richlander commented Jun 11, 2025

Uh oh!

azure-pipelines bot commented Jun 11, 2025

Uh oh!

richlander commented Jun 11, 2025

Uh oh!

richlander commented Jun 26, 2025

Uh oh!

richlander commented Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

richlander commented Jun 27, 2025

Uh oh!

richlander commented Jun 27, 2025

Uh oh!

azure-pipelines bot commented Jun 27, 2025

Uh oh!

azure-pipelines bot commented Jun 27, 2025

Uh oh!

richlander commented Jun 27, 2025

Uh oh!

richlander commented Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Inner loop

extra-platforms

outerloop

Uh oh!

jkotas commented Jun 27, 2025

Uh oh!

richlander commented Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jkotas commented Jun 27, 2025

Uh oh!

jkotas commented Jun 27, 2025

Uh oh!

richlander commented Jun 27, 2025

Uh oh!

jkotas left a comment

Choose a reason for hiding this comment

Uh oh!

richlander commented Jun 27, 2025

Uh oh!

richlander commented Jun 27, 2025

Uh oh!

azure-pipelines bot commented Jun 27, 2025

Uh oh!

azure-pipelines bot commented Jun 27, 2025

Uh oh!

richlander commented Jun 27, 2025 •

edited

Loading

richlander commented Jun 27, 2025 •

edited

Loading

richlander commented Jun 27, 2025 •

edited

Loading