AWSSDK.SimpleNotificationService - Rate exceeded

[pregress]

Describe the bug

After upgrading the AWSSDK.SimpleNotificationService package to version: 4.0.0.2
We receive the Error: Rate exceeded when running the code in an Azure Function (.NET 9, Isolated Worker, West Europe, Fucntion version: 4.1038.400.1, Linux)

Rolling back to version 3.7.400.151 immediately fixes this issue.

I'm assuming there is some new code to ratelimit by IP ?
Because running the same code localy with version: 4.0.0.2 also works, only deployed to Azure Functions it fails.

Code that fails:

public async IAsyncEnumerable<SnsTopicInfo> GetAllTopicsAsync()
{
    string nextToken = null;

    do
    {
        var result = await _snsClient.ListTopicsAsync(new ListTopicsRequest { NextToken = nextToken });
        nextToken = result.NextToken;

        foreach (var topic in result.Topics)
        {
            yield return new SnsTopicInfo { Arn = topic.TopicArn };
        }
    }
    while (!string.IsNullOrWhiteSpace(nextToken));
}

// Different method:
GetAllTopicsAsync().ToList()

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

No exception in Azure functions, since this is working locally.
I would expect rate limits based on the AwsCredentials instead of outbound IP.

Current Behavior

AmazonSimpleNotificationServiceException with status code BadRequest and ErrorCode = Throttling

Amazon.SimpleNotificationService.AmazonSimpleNotificationServiceException:
at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionStream (AWSSDK.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604)
at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler+d__2.MoveNext (AWSSDK.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1+ConfiguredTaskAwaiter.GetResult (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at Amazon.Runtime.Internal.ExceptionHandler1+d__6.MoveNext (AWSSDK.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1+ConfiguredTaskAwaiter.GetResult (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at Amazon.Runtime.Internal.ErrorHandler+<ProcessExceptionAsync>d__8.MoveNext (AWSSDK.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604) at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1+ConfiguredTaskAwaiter.GetResult (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Amazon.Runtime.Internal.ErrorHandler+d__51.MoveNext (AWSSDK.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604) at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at Amazon.Runtime.Internal.CallbackHandler+<InvokeAsync>d__91.MoveNext (AWSSDK.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Amazon.Runtime.Internal.Signer+d__11.MoveNext (AWSSDK.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604) at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at Amazon.Runtime.Internal.EndpointDiscoveryHandler+<InvokeAsync>d__21.MoveNext (AWSSDK.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Amazon.Runtime.Internal.EndpointDiscoveryHandler+d__21.MoveNext (AWSSDK.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604) at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at Amazon.Runtime.Internal.RetryHandler+<InvokeAsync>d__101.MoveNext (AWSSDK.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Amazon.Runtime.Internal.RetryHandler+d__101.MoveNext (AWSSDK.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604) at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at Amazon.Runtime.Internal.CallbackHandler+<InvokeAsync>d__91.MoveNext (AWSSDK.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Amazon.Runtime.Internal.BaseAuthResolverHandler+d__21.MoveNext (AWSSDK.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604) at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at Amazon.Runtime.Internal.CallbackHandler+<InvokeAsync>d__91.MoveNext (AWSSDK.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Amazon.Runtime.Internal.ErrorCallbackHandler+d__51.MoveNext (AWSSDK.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604) at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at Amazon.Runtime.Internal.MetricsHandler+<InvokeAsync>d__11.MoveNext (AWSSDK.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Redacted+d__5.MoveNext (Redacted.BL, Version=2025.5.12.95, Culture=neutral, PublicKeyToken=null: Redacted/AwsSnsClientService.cs:57)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore1.ThrowForFailedGetResult (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at Redacted+<GetAllTopicsAsync>d__5.System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult (Redacted.BL, Version=2025.5.12.95, Culture=neutral, PublicKeyToken=null) at Redacted+<RunAsync>d__7.MoveNext (Redacted.BL, Version=2025.5.12.95, Culture=neutral, PublicKeyToken=null: Redacted/Redacted.cs:45) at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at Redacted+<RunAsync>d__7.MoveNext (Redacted.BL, Version=2025.5.12.95, Culture=neutral, PublicKeyToken=null: Redacted/Redacted.cs:45) at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter1.GetResult (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Redacted+d__3.MoveNext (Redacted.Functions, Version=2025.5.12.95, Culture=neutral, PublicKeyToken=null: Redacted.cs:27)
Inner exception Amazon.Runtime.Internal.HttpErrorResponseException handled at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionStream:
at Amazon.Runtime.HttpWebRequestMessage.ProcessHttpResponseMessage (AWSSDK.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604)
at Amazon.Runtime.HttpWebRequestMessage+d__23.MoveNext (AWSSDK.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Amazon.Runtime.Internal.HttpHandler1+<InvokeAsync>d__101.MoveNext (AWSSDK.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Amazon.Runtime.Internal.Unmarshaller+d__31.MoveNext (AWSSDK.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604) at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at Amazon.Runtime.Internal.ErrorHandler+<InvokeAsync>d__51.MoveNext (AWSSDK.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604)

Reproduction Steps

1. Deploy an azure function v4 - isolated worker in west europe, Linux
2. Use the ListTopicsAsync method to fetch all available topics in aws SNS
3. Exception

Possible Solution

Workaround
Downgrade to 3.7.400.151

Fix
Ratelimit on credential instead of IP?

Additional Information/Context

When a ratelimit occurs I would expect the StatusCode of AmazonSimpleNotificationServiceException to be TooManyRequests and not a BadRequest.

When introducing a new Major version, some form of release notes is highly appreciated (inside of the nuget package). Consider following the best practices for nuget packages: https://learn.microsoft.com/en-us/nuget/create-packages/package-authoring-best-practices and set the Release notes.

AWS .NET SDK and/or Package version used

AWSSDK.SimpleNotificationService 4.0.0.2

Targeted .NET Platform

.NET 9

Operating System and version

Azure Functions 4.1038.400.1, Linux, West europe

[normj]

With V4 collections like the result.Topics can be null if there are no topics returned. Before the foreach (var topic in result.Topics) line there is no null check. I'm wondering if you are possibly getting a null pointer exception and then quickly doing a retry over and over causing the rate exceeded.

[pregress]

@normj no there is retry logic for NullReferenceExceptions . if Topics would be null, we would receive a NullReferenceException when iterating over it, which is not the case.

[dscpinheiro]

Do you have request IDs you could share? (Those can be found by enabling debug logging [1]).

There's no new code in V4 that rate limits service calls, so it's really odd this code fails on Azure functions but not locally. With the request IDs, we can reach out to the service team to understand what's happening.

[1] Debug logging (needs to be enabled prior to creating the service client):

AWSConfigs.LoggingConfig.LogTo = LoggingOptions.Console;
AWSConfigs.LoggingConfig.LogMetricsFormat = LogMetricsFormatOption.JSON;
AWSConfigs.LoggingConfig.LogResponses = ResponseLoggingOption.Always;
AWSConfigs.LoggingConfig.LogMetrics = true;

var _snsClient = new AmazonSimpleNotificationServiceClient();
string nextToken = null;

do
{
    var result = await _snsClient.ListTopicsAsync(new ListTopicsRequest { NextToken = nextToken });
    nextToken = result.NextToken;

    foreach (var topic in result.Topics)
    {
        Console.WriteLine(topic.TopicArn);
    }
}
while (!string.IsNullOrWhiteSpace(nextToken));

Also, this is minor and doesn't solve the problem, but the SDK has a built-in paginator for that operation (so that you don't need to handle the pagination token):

var paginator = _snsClient.Paginators.ListTopics(new ListTopicsRequest());

// Same result as above
await foreach (var topic in paginator.Topics)
{
    Console.WriteLine(topic.TopicArn);
}