Null Reference due to missing sqlAliasManager in SelectExpression

[ocdi]

Bug description

I have a query that after upgrading to EFCore 9 stopped working. The query in the main system is performing paginated syncing in chunks.

The bug appears to be in how the optimiser is handling nullability, with an IN clause having both sides nullable.

Your code

This code demonstrates the bug. The master branch is .NET9, downgrading to EFCore 8 + .NET8 the code runs (net8 branch).

https://github.com/ocdi/ef9bug

Note the code includes a bunch of extra complexity that mirrors more of what we are syncing, turns out the bug is triggered with much less in the first projection. The complexity is in how we're doing the pagination perhaps. It was difficult to trigger this code path, if I make OwnerId/OwnerType required on the Item class, the bug does not happen.

Stack traces

This is from debugging with symbols loaded

   at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.PushdownIntoSubqueryInternal(Boolean liftOrderings) in /_/src/EFCore.Relational/Query/SqlExpressions/SelectExpression.cs:line 3648
   at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.ApplyPredicate(SqlExpression sqlExpression) in /_/src/EFCore.Relational/Query/SqlExpressions/SelectExpression.cs:line 1543
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.VisitIn(InExpression inExpression, Boolean allowOptimizedExpansion, Boolean& nullable) in /_/src/EFCore.Relational/Query/SqlNullabilityProcessor.cs:line 842
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean preserveColumnNullabilityInformation, Boolean& nullable) in /_/src/EFCore.Relational/Query/SqlNullabilityProcessor.cs:line 459
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.VisitSqlBinary(SqlBinaryExpression sqlBinaryExpression, Boolean allowOptimizedExpansion, Boolean& nullable) in /_/src/EFCore.Relational/Query/SqlNullabilityProcessor.cs:line 1274
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean preserveColumnNullabilityInformation, Boolean& nullable) in /_/src/EFCore.Relational/Query/SqlNullabilityProcessor.cs:line 469
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean& nullable) in /_/src/EFCore.Relational/Query/SqlNullabilityProcessor.cs:line 427
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SelectExpression selectExpression, Boolean visitProjection) in /_/src/EFCore.Relational/Query/SqlNullabilityProcessor.cs:line 345
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SelectExpression selectExpression) in /_/src/EFCore.Relational/Query/SqlNullabilityProcessor.cs:line 290
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Process(Expression queryExpression, IReadOnlyDictionary`2 parameterValues, Boolean& canCache) in /_/src/EFCore.Relational/Query/SqlNullabilityProcessor.cs:line 85
   at Microsoft.EntityFrameworkCore.SqlServer.Query.Internal.SqlServerParameterBasedSqlProcessor.ProcessSqlNullability(Expression selectExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache)
   at Microsoft.EntityFrameworkCore.Query.RelationalParameterBasedSqlProcessor.Optimize(Expression queryExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache) in /_/src/EFCore.Relational/Query/RelationalParameterBasedSqlProcessor.cs:line 56
   at Microsoft.EntityFrameworkCore.SqlServer.Query.Internal.SqlServerParameterBasedSqlProcessor.Optimize(Expression queryExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache)
   at Microsoft.EntityFrameworkCore.Query.Internal.RelationalCommandCache.GetRelationalCommandTemplate(IReadOnlyDictionary`2 parameters) in /_/src/EFCore.Relational/Query/Internal/RelationalCommandCache.cs:line 73
   at Microsoft.EntityFrameworkCore.Internal.RelationalCommandResolverExtensions.RentAndPopulateRelationalCommand(RelationalCommandResolver relationalCommandResolver, RelationalQueryContext queryContext) in /_/src/EFCore.Relational/Extensions/Internal/RelationalCommandResolverExtensions.cs:line 27
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.InitializeReader(Enumerator enumerator) in /_/src/EFCore.Relational/Query/Internal/SingleQueryingEnumerable.cs:line 263
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.<>c.<MoveNext>b__21_0(DbContext _, Enumerator enumerator) in /_/src/EFCore.Relational/Query/Internal/SingleQueryingEnumerable.cs:line 199
   at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.MoveNext() in /_/src/EFCore.Relational/Query/Internal/SingleQueryingEnumerable.cs:line 257

This is the console output from the test project linked.

fail: Microsoft.EntityFrameworkCore.Query[10100]
      An exception occurred while iterating over the results of a query for context type 'EF9Bug.TestContext'.
      System.NullReferenceException: Object reference not set to an instance of an object.
         at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.PushdownIntoSubqueryInternal(Boolean liftOrderings)
         at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.ApplyPredicate(SqlExpression sqlExpression)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.VisitIn(InExpression inExpression, Boolean allowOptimizedExpansion, Boolean& nullable)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean preserveColumnNullabilityInformation, Boolean& nullable)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.VisitSqlBinary(SqlBinaryExpression sqlBinaryExpression, Boolean allowOptimizedExpansion, Boolean& nullable)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean preserveColumnNullabilityInformation, Boolean& nullable)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean& nullable)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SelectExpression selectExpression, Boolean visitProjection)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SelectExpression selectExpression)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Process(Expression queryExpression, IReadOnlyDictionary`2 parameterValues, Boolean& canCache)
         at Microsoft.EntityFrameworkCore.SqlServer.Query.Internal.SqlServerParameterBasedSqlProcessor.ProcessSqlNullability(Expression selectExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache)
         at Microsoft.EntityFrameworkCore.Query.RelationalParameterBasedSqlProcessor.Optimize(Expression queryExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache)
         at Microsoft.EntityFrameworkCore.SqlServer.Query.Internal.SqlServerParameterBasedSqlProcessor.Optimize(Expression queryExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache)
         at Microsoft.EntityFrameworkCore.Query.Internal.RelationalCommandCache.GetRelationalCommandTemplate(IReadOnlyDictionary`2 parameters)
         at Microsoft.EntityFrameworkCore.Internal.RelationalCommandResolverExtensions.RentAndPopulateRelationalCommand(RelationalCommandResolver relationalCommandResolver, RelationalQueryContext queryContext)
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.InitializeReader(Enumerator enumerator)
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.<>c.<MoveNext>b__21_0(DbContext _, Enumerator enumerator)
         at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.MoveNext()
      System.NullReferenceException: Object reference not set to an instance of an object.
         at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.PushdownIntoSubqueryInternal(Boolean liftOrderings)
         at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.ApplyPredicate(SqlExpression sqlExpression)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.VisitIn(InExpression inExpression, Boolean allowOptimizedExpansion, Boolean& nullable)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean preserveColumnNullabilityInformation, Boolean& nullable)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.VisitSqlBinary(SqlBinaryExpression sqlBinaryExpression, Boolean allowOptimizedExpansion, Boolean& nullable)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean preserveColumnNullabilityInformation, Boolean& nullable)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean& nullable)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SelectExpression selectExpression, Boolean visitProjection)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SelectExpression selectExpression)
         at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Process(Expression queryExpression, IReadOnlyDictionary`2 parameterValues, Boolean& canCache)
         at Microsoft.EntityFrameworkCore.SqlServer.Query.Internal.SqlServerParameterBasedSqlProcessor.ProcessSqlNullability(Expression selectExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache)
         at Microsoft.EntityFrameworkCore.Query.RelationalParameterBasedSqlProcessor.Optimize(Expression queryExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache)
         at Microsoft.EntityFrameworkCore.SqlServer.Query.Internal.SqlServerParameterBasedSqlProcessor.Optimize(Expression queryExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache)
         at Microsoft.EntityFrameworkCore.Query.Internal.RelationalCommandCache.GetRelationalCommandTemplate(IReadOnlyDictionary`2 parameters)
         at Microsoft.EntityFrameworkCore.Internal.RelationalCommandResolverExtensions.RentAndPopulateRelationalCommand(RelationalCommandResolver relationalCommandResolver, RelationalQueryContext queryContext)
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.InitializeReader(Enumerator enumerator)
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.<>c.<MoveNext>b__21_0(DbContext _, Enumerator enumerator)
         at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.MoveNext()
Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.PushdownIntoSubqueryInternal(Boolean liftOrderings)
   at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.ApplyPredicate(SqlExpression sqlExpression)
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.VisitIn(InExpression inExpression, Boolean allowOptimizedExpansion, Boolean& nullable)
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean preserveColumnNullabilityInformation, Boolean& nullable)
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.VisitSqlBinary(SqlBinaryExpression sqlBinaryExpression, Boolean allowOptimizedExpansion, Boolean& nullable)
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean preserveColumnNullabilityInformation, Boolean& nullable)
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SqlExpression sqlExpression, Boolean allowOptimizedExpansion, Boolean& nullable)
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SelectExpression selectExpression, Boolean visitProjection)
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Visit(SelectExpression selectExpression)
   at Microsoft.EntityFrameworkCore.Query.SqlNullabilityProcessor.Process(Expression queryExpression, IReadOnlyDictionary`2 parameterValues, Boolean& canCache)
   at Microsoft.EntityFrameworkCore.SqlServer.Query.Internal.SqlServerParameterBasedSqlProcessor.ProcessSqlNullability(Expression selectExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache)
   at Microsoft.EntityFrameworkCore.Query.RelationalParameterBasedSqlProcessor.Optimize(Expression queryExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache)
   at Microsoft.EntityFrameworkCore.SqlServer.Query.Internal.SqlServerParameterBasedSqlProcessor.Optimize(Expression queryExpression, IReadOnlyDictionary`2 parametersValues, Boolean& canCache)
   at Microsoft.EntityFrameworkCore.Query.Internal.RelationalCommandCache.GetRelationalCommandTemplate(IReadOnlyDictionary`2 parameters)
   at Microsoft.EntityFrameworkCore.Internal.RelationalCommandResolverExtensions.RentAndPopulateRelationalCommand(RelationalCommandResolver relationalCommandResolver, RelationalQueryContext queryContext)
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.InitializeReader(Enumerator enumerator)
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.<>c.<MoveNext>b__21_0(DbContext _, Enumerator enumerator)
   at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.MoveNext()
   at System.Linq.Enumerable.<ToArray>g__EnumerableToArray|314_0[TSource](IEnumerable`1 source)
   at Program.<Main>$(String[] args) in D:\code\EF9Bug\Program.cs:line 53

Verbose output

EF Core version

9.0.1

Database provider

SqlServer

Target framework

.NET 9.0

Operating system

No response

IDE

No response

[roji]

Confirmed, thanks for submitting @ocdi - see below for a minimal repro (it's always appreciated if the repro can be narrowed down as much as possible, like the below).

await using var context = new TestContext();
await context.Database.EnsureDeletedAsync();
await context.Database.EnsureCreatedAsync();

_ = context
    .Set<Item>()
    .Select(a => a.OwnerId!)
    .Where(a => context
        .Set<Item>()
        .Select(a => a.OwnerId)
        .Take(100)
        .Contains(a))
    .ToListAsync();

public class Item
{
    public int Id { get; set; }
    public string? OwnerId { get; set; }
}

public class TestContext : DbContext
{
    public DbSet<Item> Items { get; set; }

    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        => optionsBuilder
            .UseSqlServer("Server=localhost;Database=test;User=SA;Password=Abcd5678;Connect Timeout=60;ConnectRetryCount=0;Encrypt=false")
            .LogTo(Console.WriteLine, LogLevel.Information)
            .EnableSensitiveDataLogging();
}

[roji]

Probably the same as #35192.