How to set spark.docker.test.persistMinikube when running the integration tests?

[lins05]

When I run the k8s integration tests, the property spark.docker.test.persistMinikube seems not respected.

build/mvn -Pkubernetes -Pkubernetes-integration-tests \
          -pl resource-managers/kubernetes/integration-tests  \
          -Dtest=xxx -DwildcardSuites=org.apache.spark.deploy.kubernetes.integrationtest.KubernetesSuite \
          -Dspark.docker.test.persistMinikube=true \
          integration-test -am

But the minikube cluster is always deleted after the tests.

I also tried to run the tests with sbt, but since we haven't added sbt support for k8s modules, currently the command below doesn't work yet:

./build/sbt -Pkubernetes -Pkubernetes-integration-tests \
    -Dspark.docker.test.persistMinikube=true \
    "kubernetes-integration-tests/test-only org.apache.spark.deploy.kubernetes.integrationtest.KubernetesSuite"

[mccheah]

What are you doing to verify that the cluster is torn down?

[lins05]

@mccheah I see log messages like these:

17/01/23 07:11:44 INFO Minikube: 192.168.99.100
17/01/23 07:11:48 INFO Minikube: 192.168.99.100
17/01/23 07:13:52 INFO Minikube: minikubeVM: Running
17/01/23 07:13:52 INFO Minikube: localkube: Running
17/01/23 07:13:55 INFO Minikube: Deleting local Kubernetes cluster...
17/01/23 07:13:56 INFO Minikube: Machine deleted.

And minikube status also shows that the VM doesn't exist.

[mccheah]

Hm, from this stackoverflow post it's possible that passing system properties via -D doesn't propagate the property to the test. The test is run in a forked JVM and it doesn't inherit the system properties from the Maven process.

Looks like we can try setting -DforkMode=never or -DargLine="-Dspark.docker.test.persistMinikube=true".

[mccheah]

Filed #39 to track the sbt build.

[ssuchter]

How does the forked JVM run? Is it coming from a particular maven plugin?

[mccheah]

Don't know offhand, think it's the surefire plugin though so there may be something there.

[ssuchter]

Is there a patch or command line you are using that I can try to use? Or is this coming from some file in the tests dir? If I can see how it’s invoked within Maven, I can probably suggest the way to pass the options. Sean On January 25, 2017 at 3:12:52 PM, mccheah (notifications@github.com) wrote: Don't know offhand, think it's the surefire plugin though so there may be something there. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#37 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB2E26NQiNP8JFVhjpCD-Y10RpG7wctBks5rV9bzgaJpZM4Lq9qt> .

[mccheah]

@lins05 posted what he tried above, and it results in the Minikube VM still being deleted in the afterAll block.

[lins05]

Looks like we can try setting -DforkMode=never or -DargLine="-Dspark.docker.test.persistMinikube=true".

I tried argLine but it didn't work. I guess it's because argLine is already set in spark's top level pom.xml: https://github.com/apache/spark/blob/v2.1.0/pom.xml#L2055 so it's not possible to overwrite it from command line.

Using -DforkMode=never would cause a different error:

  java.lang.RuntimeException: Unable to load a Suite class that was discovered in the runpath: org.apache.spark.deploy.kubernetes.integrationtest.KubernetesSuite

[lins05]

Please see #52 for a possible solution.

[mccheah]

Did #52 resolve it? Can we close this?

[lins05]

Right, let's close it.