java.lang.IllegalStateException: FieldExistsQuery requires that the field indexes doc values, norms or vectors

[fang-xing-esql]

Elasticsearch Version

main

Installed Plugins

No response

Java Version

bundled

OS Version

serverless

Problem Description

org.elasticsearch.search.query.QueryPhaseExecutionException: Query Failed [Failed to execute main query]
at org.elasticsearch.server@9.1.0/org.elasticsearch.search.query.QueryPhase.addCollectorsAndSearch(QueryPhase.java:237)
at org.elasticsearch.server@9.1.0/org.elasticsearch.search.query.QueryPhase.executeQuery(QueryPhase.java:143)
at org.elasticsearch.server@9.1.0/org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:70)
at org.elasticsearch.server@9.1.0/org.elasticsearch.indices.IndicesService.lambda$loadIntoContext$39(IndicesService.java:1669)
at org.elasticsearch.server@9.1.0/org.elasticsearch.indices.IndicesService.lambda$cacheShardLevelResult$40(IndicesService.java:1735)
at org.elasticsearch.server@9.1.0/org.elasticsearch.indices.IndicesRequestCache$Loader.load(IndicesRequestCache.java:165)
at org.elasticsearch.server@9.1.0/org.elasticsearch.indices.IndicesRequestCache$Loader.load(IndicesRequestCache.java:148)
at org.elasticsearch.server@9.1.0/org.elasticsearch.common.cache.Cache.computeIfAbsent(Cache.java:440)
at org.elasticsearch.server@9.1.0/org.elasticsearch.indices.IndicesRequestCache.getOrCompute(IndicesRequestCache.java:111)
at org.elasticsearch.server@9.1.0/org.elasticsearch.indices.IndicesService.cacheShardLevelResult(IndicesService.java:1741)
at org.elasticsearch.server@9.1.0/org.elasticsearch.indices.IndicesService.loadIntoContext(IndicesService.java:1663)
at org.elasticsearch.server@9.1.0/org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:670)
at org.elasticsearch.server@9.1.0/org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:877)
at org.elasticsearch.server@9.1.0/org.elasticsearch.search.SearchService.lambda$executeQueryPhase$7(SearchService.java:710)
at org.elasticsearch.server@9.1.0/org.elasticsearch.action.ActionRunnable$3.accept(ActionRunnable.java:79)
at org.elasticsearch.server@9.1.0/org.elasticsearch.action.ActionRunnable$3.accept(ActionRunnable.java:76)
at org.elasticsearch.server@9.1.0/org.elasticsearch.action.ActionRunnable$4.doRun(ActionRunnable.java:101)
at org.elasticsearch.server@9.1.0/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)
at org.elasticsearch.server@9.1.0/org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:35)
at org.elasticsearch.server@9.1.0/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:1044)
at org.elasticsearch.server@9.1.0/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1095)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:619)
at java.base/java.lang.Thread.run(Thread.java:1447)
Caused by: java.lang.IllegalStateException: FieldExistsQuery requires that the field indexes doc values, norms or vectors, but field 'falco.output' exists and indexes neither of these data structures
at org.apache.lucene.core@10.2.1/org.apache.lucene.search.FieldExistsQuery.rewrite(FieldExistsQuery.java:156)
at org.apache.lucene.core@10.2.1/org.apache.lucene.search.ConstantScoreQuery.rewrite(ConstantScoreQuery.java:44)
at org.apache.lucene.core@10.2.1/org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:311)
at org.apache.lucene.core@10.2.1/org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:316)
at org.apache.lucene.core@10.2.1/org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:876)
at org.elasticsearch.server@9.1.0/org.elasticsearch.search.internal.ContextIndexSearcher.rewrite(ContextIndexSearcher.java:205)
at org.elasticsearch.server@9.1.0/org.elasticsearch.search.internal.SearchContext.rewrittenQuery(SearchContext.java:296)
at org.elasticsearch.server@9.1.0/org.elasticsearch.search.query.QueryPhase.addCollectorsAndSearch(QueryPhase.java:164)

Steps to Reproduce

Serverless alert - Found 50 data node errors of type: java.lang.IllegalStateException.

Logs (if relevant)

No response

[elasticsearchmachine]

Pinging @elastic/es-search (Team:Search)

[elasticsearchmachine]

Pinging @elastic/es-search-relevance (Team:Search Relevance)

[benwtrent]

yeah, we should catch and throw bad_request here. its possible for the user to fix, and a symptom of their specific settings.

[pmpailis]

I've been trying to replicate this but haven't been successful so far. Should we catch all IllegalStateException during ContentIndexSearcher#rewrite and throw them as bad request? I guess it means that there is a mismatch between the query and index, but should we reroute this to a 4xx ?