requirement violation in BatchingExecutor

[scabug]

The Play team found this flaw in Akka’s version of the BatchingExecutor, see akka/akka#17390 and playframework/playframework#4372.

[scabug]

Imported From: https://issues.scala-lang.org/browse/SI-9304?orig=1
Reporter: @rkuhn

[scabug]

@retronym said (edited on May 13, 2015 2:03:15 AM UTC):
@rkuhn I've tried to bring this patch across but I'm stuck. I must admit I still find the internals of scala.concurrent._ a little hard to navigate. Could you take a look and see what I'm doing wrong?

scala/scala@2.11.x...retronym:ticket/9304

java.lang.IllegalArgumentException: requirement failed
	at scala.Predef$.require(Predef.scala:207)
	at scala.concurrent.BatchingExecutor$Batch.run(BatchingExecutor.scala:52)
	at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:599)
	at scala.concurrent.BatchingExecutor$Batch.blockOn(BatchingExecutor.scala:92)
	at scala.concurrent.package$.blocking(package.scala:123)
	at BlockContexts_t9304$$anon$5$$anonfun$77$$anonfun$27.apply$mcVI$sp(scala-concurrent-tck.scala:586)
	at BlockContexts_t9304$$anon$5$$anonfun$77$$anonfun$27.apply(scala-concurrent-tck.scala:584)
	at BlockContexts_t9304$$anon$5$$anonfun$77$$anonfun$27.apply(scala-concurrent-tck.scala:584)
	at scala.util.Success$$anonfun$map$1.apply(Try.scala:236)
	at scala.util.Try$.apply(Try.scala:191)
	at scala.util.Success.map(Try.scala:236)
	at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235)
	at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
	at scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:63)
	at scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:78)
	at scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:57)
	at scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:57)
	at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
	at scala.concurrent.BatchingExecutor$Batch.run(BatchingExecutor.scala:56)
	at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:599)
	at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:105)
	at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:597)
	at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:40)
	at scala.concurrent.impl.Promise$DefaultPromise.scala$concurrent$impl$Promise$DefaultPromise$$dispatchOrAddCallback(Promise.scala:280)
	at scala.concurrent.impl.Promise$DefaultPromise.onComplete(Promise.scala:270)
	at scala.concurrent.Future$class.flatMap(Future.scala:249)
	at scala.concurrent.impl.Promise$DefaultPromise.flatMap(Promise.scala:153)
	at BlockContexts_t9304$$anon$5.run(scala-concurrent-tck.scala:579)
	at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:599)
	at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:108)
	at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:597)
	at BlockContexts_t9304$class.testTaskUsingBlockingMultipleTimes(scala-concurrent-tck.scala:577)
	at Test$.testTaskUsingBlockingMultipleTimes(scala-concurrent-tck.scala:813)

[scabug]

@rkuhn said:
Thanks for looking into this, Jason! I was planning on assessing the current situation in order to figure out whether this fix is needed or not (since both code bases share a common ancestor here), and it seems that no change is required beyond adding the test case. The test is wrong, BTW, because the used ExecutionContext must be one that makes use of the BatchingExecutor and supports asynchronous execution, so Future.InternalCallbackExecutor will not fit the bill. You could create a BatchingExecutor that delegates unbatchedExecute to the common pool and test with that one. If the test passes without other code modifications then we’re good.

One more thing: BatchingExecutor#72 might want to check if there actually are remaining tasks.

[scabug]

Henning Wielenberg (henning.wielenberg) said:
Hi,

i somehow managed to get this exception in our project. But i cant give you any code at this point because it's spread across multiple components and i don't even know what is responsible for it.
The exception pops up everytime while executing an integration test.

Versions:
scala 2.11.7
play 2.4.3

Here is the stack trace:

java.lang.IllegalArgumentException: requirement failed
	at scala.Predef$.require(Predef.scala:207)
	at scala.concurrent.BatchingExecutor$Batch.run(BatchingExecutor.scala:51)
	at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:599)
	at scala.concurrent.BatchingExecutor$Batch.blockOn(BatchingExecutor.scala:92)
	at scala.concurrent.package$.blocking(package.scala:123)
	at play.api.libs.iteratee.Enumerator$$anonfun$fromStream$2.apply(Enumerator.scala:562)
	at play.api.libs.iteratee.Enumerator$$anonfun$fromStream$2.apply(Enumerator.scala:560)
	at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
	at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
	at play.api.libs.iteratee.Execution$trampoline$.executeScheduled(Execution.scala:120)
	at play.api.libs.iteratee.Execution$trampoline$.execute(Execution.scala:71)
	at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:40)
	at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:248)
	at scala.concurrent.Promise$class.complete(Promise.scala:55)
	at scala.concurrent.impl.Promise$DefaultPromise.complete(Promise.scala:153)
	at scala.concurrent.Future$$anonfun$flatMap$1$$anonfun$apply$3.apply(Future.scala:254)
	at scala.concurrent.Future$$anonfun$flatMap$1$$anonfun$apply$3.apply(Future.scala:254)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
	at scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:63)
	at scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:78)
	at scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:55)
	at scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:55)
	at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
	at scala.concurrent.BatchingExecutor$Batch.run(BatchingExecutor.scala:54)
	at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:599)
	at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:106)
	at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:597)
	at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:40)
	at scala.concurrent.impl.Promise$DefaultPromise.scala$concurrent$impl$Promise$DefaultPromise$$dispatchOrAddCallback(Promise.scala:280)
	at scala.concurrent.impl.Promise$DefaultPromise.onComplete(Promise.scala:270)
	at scala.concurrent.Promise$class.tryCompleteWith(Promise.scala:76)
	at scala.concurrent.impl.Promise$DefaultPromise.tryCompleteWith(Promise.scala:153)
	at scala.concurrent.Promise$class.completeWith(Promise.scala:69)
	at scala.concurrent.impl.Promise$DefaultPromise.completeWith(Promise.scala:153)
	at controllers.SelfPopulatingMap.putIfAbsent(Assets.scala:62)
	at controllers.Assets$.assetInfo(Assets.scala:296)
	at controllers.Assets$.assetInfoForRequest(Assets.scala:302)
	at controllers.AssetsBuilder$$anonfun$17.apply(Assets.scala:441)
	at controllers.AssetsBuilder$$anonfun$17.apply(Assets.scala:440)
	at scala.Option.map(Option.scala:146)
	at controllers.AssetsBuilder.controllers$AssetsBuilder$$assetAt(Assets.scala:440)
	at controllers.AssetsBuilder$$anonfun$at$1.apply(Assets.scala:434)
	at controllers.AssetsBuilder$$anonfun$at$1.apply(Assets.scala:433)
	at play.api.mvc.Action$.invokeBlock(Action.scala:533)
	at play.api.mvc.Action$.invokeBlock(Action.scala:530)
	at play.api.mvc.ActionBuilder$$anon$1.apply(Action.scala:493)
	at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4$$anonfun$apply$5.apply(Action.scala:105)
	at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4$$anonfun$apply$5.apply(Action.scala:105)
	at play.utils.Threads$.withContextClassLoader(Threads.scala:21)
	at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4.apply(Action.scala:104)
	at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4.apply(Action.scala:103)
	at scala.Option.map(Option.scala:146)
	at play.api.mvc.Action$$anonfun$apply$1.apply(Action.scala:103)
	at play.api.mvc.Action$$anonfun$apply$1.apply(Action.scala:96)
	at play.api.libs.iteratee.DoneIteratee$$anonfun$mapM$2.apply(Iteratee.scala:741)
	at play.api.libs.iteratee.DoneIteratee$$anonfun$mapM$2.apply(Iteratee.scala:741)
	at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
	at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

[MihaiBogdanEugen]

Any news about this topic?

[SethTisue]

has anyone tried it in Scala 2.13.6 to see if the issue even still exists?

[MihaiBogdanEugen]

To be fair, we did not. We're planning to upgrade to Scala 2.13.latest next month, but right now, we're facing this issue using Scala 2.12.12 and the latest Akka version.

[SethTisue]

A lot of changes were made to scala.concurrent for 2.13. Nobody is actively working on the 2.12 version anymore, but pull requests to 2.12 will still be considered, if submitted.

[som-snytt]

I almost understand rkuhn's previous remark, that your BatchingExecutor must be able to delegate to an executor that will run jobs asynchronously. There is a WARNING in Akka. I submitted a PR on 2.12 but hadn't got around to looking at 2.13, so I don't know if backporting would make more sense.