Skip to content

test: configurations that work with graalvm for jdk 21 #1448

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 22 commits into from

Conversation

mpeddada1
Copy link
Contributor

@mpeddada1 mpeddada1 commented Aug 8, 2024

For #1447

Adding -H:-AddAllFileSystemProviders as a workaround as described in oracle/graal#5134 (comment) and running mvn test -Pnative results in a successful build for graalvm for jdk 21.

openjdk version "21.0.2" 2024-01-16
OpenJDK Runtime Environment GraalVM CE 21.0.2+13.1 (build 21.0.2+13-jvmci-23.1-b30)
OpenJDK 64-Bit Server VM GraalVM CE 21.0.2+13.1 (build 21.0.2+13-jvmci-23.1-b30, mixed mode, sharing)

@product-auto-label product-auto-label bot added size: xs Pull request size is extra small. api: storage Issues related to the googleapis/java-storage-nio API. labels Aug 8, 2024
@product-auto-label product-auto-label bot added size: s Pull request size is small. and removed size: xs Pull request size is extra small. labels Aug 19, 2024
@mpeddada1
Copy link
Contributor Author

mpeddada1 commented Aug 19, 2024

Build is successful but running into the following runtime issue for ITGcsNio:

=> org.opentest4j.MultipleFailuresError: Multiple Failures (2 failures)
	com.google.cloud.storage.StorageException: [email protected] does not have storage.buckets.create access to the Google Cloud project. Permission 'storage.buckets.create' denied on resource (or it may not exist).
	java.util.concurrent.ExecutionException: com.google.cloud.storage.StorageException: The specified bucket does not exist.
         com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:330)
         com.google.cloud.storage.spi.v1.HttpStorageRpc.create(HttpStorageRpc.java:372)
         com.google.cloud.storage.StorageImpl.lambda$create$0(StorageImpl.java:139)
         com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:102)
         com.google.cloud.RetryHelper.run(RetryHelper.java:76)
         [...]
       Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
POST [https://storage.googleapis.com/storage/v1/b?project=gcloud-devel&projection=full](https://www.google.com/url?q=https://storage.googleapis.com/storage/v1/b?project%3Dgcloud-devel%26projection%3Dfull&sa=D)
{
  "code" : 403,
  "errors" : [ {
    "domain" : "global",
    "message" : "[email protected] does not have storage.buckets.create access to the Google Cloud project. Permission 'storage.buckets.create' denied on resource (or it may not exist).",
    "reason" : "forbidden"
  } ],
  "message" : "[email protected] does not have storage.buckets.create access to the Google Cloud project. Permission 'storage.buckets.create' denied on resource (or it may not exist)."
}
         com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118)
         com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37)
         com.google.api.client.googleapis.services.AbstractGoogleClientRequest$3.interceptResponse(AbstractGoogleClientRequest.java:479)
         com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)
         com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:565)
         [...]
         [email protected]/java.util.concurrent.FutureTask.report([FutureTask.java:122](https://cs.corp.google.com/#search/&sq=package:%5Epiper$%20project:openjdk21&q=java/util/concurrent/FutureTask.java:122))
         [email protected]/java.util.concurrent.FutureTask.get([FutureTask.java:205](https://cs.corp.google.com/#search/&sq=package:%5Epiper$%20project:openjdk21&q=java/util/concurrent/FutureTask.java:205))
         com.google.cloud.storage.testing.RemoteStorageHelper.forceDelete(RemoteStorageHelper.java:159)
         com.google.cloud.storage.contrib.nio.it.ITGcsNio.afterClass(ITGcsNio.java:151)
         [email protected]/java.lang.reflect.Method.invoke([Method.java:580](https://cs.corp.google.com/#search/&sq=package:%5Epiper$%20project:openjdk21&q=java/lang/reflect/Method.java:580))
       Caused by: com.google.cloud.storage.StorageException: The specified bucket does not exist.
         com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:330)
         com.google.cloud.storage.spi.v1.HttpStorageRpc.list(HttpStorageRpc.java:475)
         com.google.cloud.storage.StorageImpl.lambda$listBlobs$9(StorageImpl.java:479)
         com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:102)
         com.google.cloud.RetryHelper.run(RetryHelper.java:76)
         [...]
       Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 404 Not Found
GET [https://storage.googleapis.com/storage/v1/b/gcloud-test-bucket-temp-3bfbf591-9851-4045-8155-3c32895491b7/o?projection=full&userProject=gcloud-devel&versions=true](https://www.google.com/url?q=https://storage.googleapis.com/storage/v1/b/gcloud-test-bucket-temp-3bfbf591-9851-4045-8155-3c32895491b7/o?projection%3Dfull%26userProject%3Dgcloud-devel%26versions%3Dtrue&sa=D)
{
  "code" : 404,
  "errors" : [ {
    "domain" : "global",
    "message" : "The specified bucket does not exist.",
    "reason" : "notFound"
  } ],
  "message" : "The specified bucket does not exist."
}
         com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118)
         com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37)
         com.google.api.client.googleapis.services.AbstractGoogleClientRequest$3.interceptResponse(AbstractGoogleClientRequest.java:479)
         com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)
         com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:565)
         [...]

Locally running mvn clean verify -Dsurefire.failIfNoSpecifiedTests=false -Dtest=ITGcsNio#testCopy_replaceFile_withoutOption -Pnative works fine

@mpeddada1 mpeddada1 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Aug 19, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Aug 19, 2024
@mpeddada1
Copy link
Contributor Author

No changes to https://github.com/googleapis/java-storage-nio/blob/416fc234e06ef6c42f0ecfdd51004537d9fb29ed/.kokoro/presubmit/graalvm-native.cfg have been made recently. GOOGLE_APPLICATION_CREDENTIALS is set to secret_manager/java-it-service-account

@mpeddada1 mpeddada1 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Aug 20, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Aug 20, 2024
@mpeddada1
Copy link
Contributor Author

mpeddada1 commented Aug 20, 2024

The standard java and graalvm 22.3.x jdk 17 tests are using: [email protected]
The graalvm for jdk 21 test is using compute engine credentials [email protected]

As per https://github.com/googleapis/google-auth-library-java?tab=readme-ov-file#getting-application-default-credentials, the prioritization order of credentials is as follows:

  • Credentials file pointed to by the GOOGLE_APPLICATION_CREDENTIALS environment variable
  • Credentials provided by the Google Cloud SDK gcloud auth application-default login command
  • Google App Engine built-in credentials
  • Google Cloud Shell built-in credentials
  • Google Compute Engine built-in credentials
    • Skip this check by setting the environment variable NO_GCE_CHECK=true
    • Customize the GCE metadata server address by setting the environment variable GCE_METADATA_HOST=

@product-auto-label product-auto-label bot added size: m Pull request size is medium. and removed size: s Pull request size is small. labels Aug 20, 2024
@mpeddada1 mpeddada1 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 1, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 1, 2024
@mpeddada1
Copy link
Contributor Author

Passing in RemoteStorageHelper gcsHelper = RemoteStorageHelper.create(System.getenv("GOOGLE_CLOUD_PROJECT"), new FileInputStream(System.getenv("GOOGLE_APPLICATION_CREDENTIALS"))); directly instead of relying on DefaultCredentialsProvider to read the configs works

@sydney-munro
Copy link
Contributor

closing this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the googleapis/java-storage-nio API. size: m Pull request size is medium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants