AccessViolationException on macOS in Asn1Tag.TryDecode

[hach-que]

Description

I suspect this is a JIT issue, similar to #88288. However in this case, turning off ready-to-run does not fix it (as it is already turned off). The stack trace I'm seeing is:

Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at System.Formats.Asn1.Asn1Tag.TryDecode(System.ReadOnlySpan`1<Byte>, System.Formats.Asn1.Asn1Tag ByRef, Int32 ByRef)
   at System.Formats.Asn1.AsnDecoder.ReadEncodedValue(System.ReadOnlySpan`1<Byte>, System.Formats.Asn1.AsnEncodingRules, Int32 ByRef, Int32 ByRef, Int32 ByRef)
   at System.Security.Cryptography.X509Certificates.CertificateData..ctor(Byte[])
   at System.Security.Cryptography.X509Certificates.AppleCertificatePal.EnsureCertData()
   at System.Security.Cryptography.X509Certificates.AppleCertificatePal.GetPolicyData()
   at System.Security.Cryptography.X509Certificates.CertificatePolicyChain.ReadPolicy(System.Security.Cryptography.X509Certificates.X509Certificate2)
   at System.Security.Cryptography.X509Certificates.CertificatePolicyChain.ReadPolicies(System.Collections.Generic.List`1<System.Security.Cryptography.X509Certificates.X509Certificate2>)
   at System.Security.Cryptography.X509Certificates.SecTrustChainPal.IsPolicyMatch(System.ValueTuple`2<System.Security.Cryptography.X509Certificates.X509Certificate2,Int32>[], System.Security.Cryptography.OidCollection, System.Security.Cryptography.OidCollection)
   at System.Security.Cryptography.X509Certificates.SecTrustChainPal.Execute(System.DateTime, Boolean, System.Security.Cryptography.OidCollection, System.Security.Cryptography.OidCollection, System.Security.Cryptography.X509Certificates.X509RevocationFlag)
   at System.Security.Cryptography.X509Certificates.ChainPal.BuildChain(Boolean, System.Security.Cryptography.X509Certificates.ICertificatePal, System.Security.Cryptography.X509Certificates.X509Certificate2Collection, System.Security.Cryptography.OidCollection, System.Security.Cryptography.OidCollection, System.Security.Cryptography.X509Certificates.X509RevocationMode, System.Security.Cryptography.X509Certificates.X509RevocationFlag, System.Security.Cryptography.X509Certificates.X509Certificate2Collection, System.Security.Cryptography.X509Certificates.X509ChainTrustMode, System.DateTime, System.TimeSpan, Boolean)
   at System.Security.Cryptography.X509Certificates.X509Chain.Build(System.Security.Cryptography.X509Certificates.X509Certificate2, Boolean)
   at System.Net.CertificateValidationPal.VerifyCertificateProperties(System.Net.Security.SafeDeleteContext, System.Security.Cryptography.X509Certificates.X509Chain, System.Security.Cryptography.X509Certificates.X509Certificate2, Boolean, Boolean, System.String)
   at System.Net.Security.SslStream.VerifyRemoteCertificate(System.Net.Security.RemoteCertificateValidationCallback, System.Net.Security.SslCertificateTrust, System.Net.Security.ProtocolToken ByRef, System.Net.Security.SslPolicyErrors ByRef, System.Security.Cryptography.X509Certificates.X509ChainStatusFlags ByRef)
   at System.Net.Security.SslStream.CompleteHandshake(System.Net.Security.ProtocolToken ByRef, System.Net.Security.SslPolicyErrors ByRef, System.Security.Cryptography.X509Certificates.X509ChainStatusFlags ByRef)
   at System.Net.Security.SslStream.CompleteHandshake(System.Net.Security.SslAuthenticationOptions)
   at System.Net.Security.SslStream+<ForceAuthenticationAsync>d__150`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Net.Security.SslStream+<ForceAuthenticationAsync>d__150`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]], System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].ExecutionContextCallback(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Net.Security.SslStream+<ForceAuthenticationAsync>d__150`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]], System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext(System.Threading.Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Net.Security.SslStream+<ForceAuthenticationAsync>d__150`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]], System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Runtime.CompilerServices.IAsyncStateMachineBox, Boolean)
   at System.Threading.Tasks.Task.RunContinuations(System.Object)
   at System.Threading.Tasks.Task`1[[System.Int32, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].TrySetResult(Int32)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.Int32, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].SetExistingTaskResult(System.Threading.Tasks.Task`1<Int32>, Int32)
   at System.Net.Security.SslStream+<ReceiveHandshakeFrameAsync>d__151`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Int32, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Net.Security.SslStream+<ReceiveHandshakeFrameAsync>d__151`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]], System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].ExecutionContextCallback(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Int32, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Net.Security.SslStream+<ReceiveHandshakeFrameAsync>d__151`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]], System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext(System.Threading.Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Int32, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Net.Security.SslStream+<ReceiveHandshakeFrameAsync>d__151`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]], System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext()
   at System.Net.Security.SslStream+<EnsureFullTlsFrameAsync>d__161`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext()
   at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1+StateMachineBox`1[[System.Int32, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Net.Security.SslStream+<EnsureFullTlsFrameAsync>d__161`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]], System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].ExecutionContextCallback(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1+StateMachineBox`1[[System.Int32, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Net.Security.SslStream+<EnsureFullTlsFrameAsync>d__161`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]], System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext()
   at System.Net.Sockets.SocketAsyncEventArgs.TransferCompletionCallbackCore(Int32, System.Memory`1<Byte>, System.Net.Sockets.SocketFlags, System.Net.Sockets.SocketError)
   at System.Net.Sockets.SocketAsyncEngine.System.Threading.IThreadPoolWorkItem.Execute()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()
   at System.Threading.Thread.StartCallback()

Reproduction Steps

This seems to randomly occur when making lots of TCP connections to the same macOS process. Other than that, I don't have an explicit list of reproduction steps.

If there's a debugger or tool I can attach to the process to get more information as to the cause of the crash, let me know.

Expected behavior

Should not see AccessViolationException.

Actual behavior

The application crashes.

Regression?

No response

Known Workarounds

No known workaround.

Configuration

macOS 12.6 M1, .NET 8 self-contained trimmed executable (but not ready-to-run)

Other information

No response

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-security, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[JulieLeeMSFT]

@EgorBo, please take a look.

[EgorBo]

Is it possible to build a repro project or at least share a crash dump? I don't think we can diagnose this AVE by only the provided stack trace

[dotnet-policy-service]

This issue has been marked needs-author-action and may be missing some important information.

[hach-que]

I think this can be closed. I believe the root cause was using libgit2sharp on macOS ARM; that native library seems to cause memory corruption that then breaks .NET code later. Since we weren't really using the Git code paths, I've just removed usage of libgit2sharp and the crashes have gone away.