Skip to content

feat(storage): add samples for soft delete (objects) #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 19 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions pubsub/api/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,67 @@ Usage: create_topic.php $projectId $topicName
@param string $topicName The Pub/Sub topic name.
```

## PHPUnit Tests

At this time, the GitHub actions in this repo fail to run the tests written in this folder. The developer is responsible for locally running and confirming their samples and corresponding tests.

### PubSub Emulator
Some tests in the pubsubTest.php requires PubSub emulator. These tests start with ```$this->requireEnv('PUBSUB_EMULATOR_HOST')```.

#### Prerequisites
- Python
```
xcode-select --install
brew install pyenv
pyenv install <version>
python3 --version
```
- JDK
```
brew install openjdk
export JAVA_HOME=<path to openjdk folder>
export PATH="$JAVA_HOME/bin:$PATH"
```

Once python, JDK, and GCloud CLI are installed, follow [these instructions](https://cloud.google.com/pubsub/docs/emulator) to run the emulator.

### Setting up environment variables
Open a new tab in terminal, separate from the one running your emulator.

```
// php-docs-samples/testing folder
$ cd ../../../testing

$ export GOOGLE_PROJECT_ID=<project id>
$ export GOOGLE_PUBSUB_TOPIC=<topic name>
$ export GOOGLE_PUBSUB_STORAGE_BUCKET=<bucket name>
$ export GOOGLE_PUBSUB_SUBSCRIPTION=<subscription name>

// only set if your test requires the emulator
$ export PUBSUB_EMULATOR_HOST=localhost:<emulator port>

// unset the PUBSUB emulator host variable if you want to run a test that doesn't require an emulator
$ unset PUBSUB_EMULATOR
```

### Running the tests
Run your test(s) like so in the same terminal tab that you set your env variables in the previous step.

```
// Run all tests in pubsubTest.php. --verbose tag is recommended to see any issues or stack trace
$ php-docs-samples/testing/vendor/bin/phpunit ../pubsub/api/test/pubsubTest.php --verbose

// Run a single test in pubsubTest.php
$ php-docs-samples/testing/vendor/bin/phpunit ../pubsub/api/test/pubsubTest.php --filter testSubscriptionPolicy --verbose
```

## Fixing Styling Errors
If you create a PR and the Lint / styles (pull_request) check fails, this is a quick fix.

```
$ php-docs-samples/testing/vendor/bin/php-cs-fixer fix <path to your file>
```

## Troubleshooting

If you get the following error, set the environment variable `GCLOUD_PROJECT` to your project ID:
Expand Down
71 changes: 71 additions & 0 deletions pubsub/api/src/create_topic_with_aws_msk_ingestion.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
<?php

/**
* Copyright 2025 Google LLC.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

/**
* For instructions on how to run the full sample:
*
* @see https://github.com/GoogleCloudPlatform/php-docs-samples/blob/main/pubsub/api/README.md
*/

namespace Google\Cloud\Samples\PubSub;

# [START pubsub_create_topic_with_aws_msk_ingestion]
use Google\Cloud\PubSub\PubSubClient;

/**
* Creates a Pub/Sub topic with AWS MSK ingestion.
*
* @param string $projectId The Google project ID.
* @param string $topicName The Pub/Sub topic name.
* @param string $clusterArn The Amazon Resource Name (ARN) that uniquely identifies the cluster.
* @param string $mskTopic The name of the topic in the Amazon MSK cluster that Pub/Sub will import from.
* @param string $awsRoleArn AWS role ARN to be used for Federated Identity authentication with Amazon MSK.
* Check the Pub/Sub docs for how to set up this role and the required permissions that need to be
* attached to it.
* @param string $gcpServiceAccount The GCP service account to be used for Federated Identity authentication
* with Amazon MSK (via a AssumeRoleWithWebIdentity call for the provided role). The aws_role_arn
* must be set up with accounts.google.com:sub equals to this service account number.
*/
function create_topic_with_aws_msk_ingestion(
string $projectId,
string $topicName,
string $clusterArn,
string $mskTopic,
string $awsRoleArn,
string $gcpServiceAccount
): void {
$pubsub = new PubSubClient([
'projectId' => $projectId,
]);

$topic = $pubsub->createTopic($topicName, [
'ingestionDataSourceSettings' => [
'aws_msk' => [
'cluster_arn' => $clusterArn,
'topic' => $mskTopic,
'aws_role_arn' => $awsRoleArn,
'gcp_service_account' => $gcpServiceAccount
]
]
]);

printf('Topic created: %s' . PHP_EOL, $topic->name());
}
# [END pubsub_create_topic_with_aws_msk_ingestion]
require_once __DIR__ . '/../../../testing/sample_helpers.php';
\Google\Cloud\Samples\execute_sample(__FILE__, __NAMESPACE__, $argv);
91 changes: 91 additions & 0 deletions pubsub/api/src/create_topic_with_cloud_storage_ingestion.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
<?php

/**
* Copyright 2025 Google LLC.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

/**
* For instructions on how to run the full sample:
*
* @see https://github.com/GoogleCloudPlatform/php-docs-samples/blob/main/pubsub/api/README.md
*/

namespace Google\Cloud\Samples\PubSub;

# [START pubsub_create_topic_with_cloud_storage_ingestion]
use Google\Cloud\PubSub\PubSubClient;
use Google\Cloud\PubSub\V1\IngestionDataSourceSettings\CloudStorage\AvroFormat;
use Google\Cloud\PubSub\V1\IngestionDataSourceSettings\CloudStorage\PubSubAvroFormat;
use Google\Cloud\PubSub\V1\IngestionDataSourceSettings\CloudStorage\TextFormat;
use Google\Protobuf\Timestamp;

/**
* Creates a topic with Cloud Storage Ingestion.
*
* @param string $projectId The Google project ID.
* @param string $topicName The Pub/Sub topic name.
* @param string $bucket Cloud Storage bucket.
* @param string $inputFormat Input format for the Cloud Storage data. Must be one of text, avro, or pubsub_avro.
* @param string $minimumObjectCreatedTime Only objects with a larger or equal creation timestamp will be ingested.
* @param string $textDelimiter Delimiter for text format input.
* @param string $matchGlob Glob pattern used to match objects that will be ingested. If unset, all objects will be ingested.
*/
function create_topic_with_cloud_storage_ingestion(
string $projectId,
string $topicName,
string $bucket,
string $inputFormat,
string $minimumObjectCreatedTime,
string $textDelimiter = '',
string $matchGlob = ''
): void {
$datetime = new \DateTimeImmutable($minimumObjectCreatedTime);
$timestamp = (new Timestamp())
->setSeconds($datetime->getTimestamp())
->setNanos($datetime->format('u') * 1000);

$cloudStorageData = [
'bucket' => $bucket,
'minimum_object_create_time' => $timestamp
];

$cloudStorageData[$inputFormat . '_format'] = match($inputFormat) {
'text' => new TextFormat(['delimiter' => $textDelimiter]),
'avro' => new AvroFormat(),
'pubsub_avro' => new PubSubAvroFormat(),
default => throw new \InvalidArgumentException(
'inputFormat must be in (\'text\', \'avro\', \'pubsub_avro\'); got value: ' . $inputFormat
)
};

if (!empty($matchGlob)) {
$cloudStorageData['match_glob'] = $matchGlob;
}

$pubsub = new PubSubClient([
'projectId' => $projectId,
]);

$topic = $pubsub->createTopic($topicName, [
'ingestionDataSourceSettings' => [
'cloud_storage' => $cloudStorageData
]
]);

printf('Topic created: %s' . PHP_EOL, $topic->name());
}
# [END pubsub_create_topic_with_cloud_storage_ingestion]
require_once __DIR__ . '/../../../testing/sample_helpers.php';
\Google\Cloud\Samples\execute_sample(__FILE__, __NAMESPACE__, $argv);
57 changes: 57 additions & 0 deletions pubsub/api/src/create_unwrapped_push_subscription.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
<?php

/**
* Copyright 2025 Google LLC.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

/**
* For instructions on how to run the full sample:
*
* @see https://github.com/GoogleCloudPlatform/php-docs-samples/blob/main/pubsub/api/README.md
*/

namespace Google\Cloud\Samples\PubSub;

# [START pubsub_create_unwrapped_push_subscription]
use Google\Cloud\PubSub\PubSubClient;

/**
* Create unwrappped push subscription.
*
* @param string $projectId The Google project ID.
* @param string $topicName The Pub/Sub topic name.
* @param string $subscriptionId The ID of the subscription.
*/
function create_unwrapped_push_subscription(
string $projectId,
string $topicName,
string $subscriptionId
): void {

$pubsub = new PubSubClient([
'projectId' => $projectId,
]);
$pubsub->subscribe($subscriptionId, $topicName, [
'pushConfig' => [
'no_wrapper' => [
'write_metadata' => true
]
]
]);
printf('Unwrapped push subscription created: %s' . PHP_EOL, $subscriptionId);
}
# [END pubsub_create_unwrapped_push_subscription]
require_once __DIR__ . '/../../../testing/sample_helpers.php';
\Google\Cloud\Samples\execute_sample(__FILE__, __NAMESPACE__, $argv);
66 changes: 66 additions & 0 deletions pubsub/api/src/optimistic_subscribe.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
<?php

/**
* Copyright 2025 Google LLC.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

/**
* For instructions on how to run the full sample:
*
* @see https://github.com/GoogleCloudPlatform/php-docs-samples/blob/main/pubsub/api/README.md
*/

namespace Google\Cloud\Samples\PubSub;

# [START pubsub_optimistic_subscribe]
use Google\Cloud\Core\Exception\NotFoundException;
use Google\Cloud\PubSub\PubSubClient;

/**
* Optimistically subscribes to a topic
*
* @param string $projectId The Google project ID.
* @param string $topicName The Pub/Sub topic name.
* @param string $subscriptionId The ID of the subscription.
*/
function optimistic_subscribe(
string $projectId,
string $topicName,
string $subscriptionId
): void {

$pubsub = new PubSubClient([
'projectId' => $projectId,
]);

$subscription = $pubsub->subscription($subscriptionId);

try {
$messages = $subscription->pull();
foreach ($messages as $message) {
printf('PubSub Message: %s' . PHP_EOL, $message->data());
$subscription->acknowledge($message);
}
} catch (NotFoundException $e) { // Subscription is not found
printf('Exception Message: %s' . PHP_EOL, $e->getMessage());
printf('StackTrace: %s' . PHP_EOL, $e->getTraceAsString());
// Create subscription and retry the pull. Any messages published before subscription creation would not be received.
$pubsub->subscribe($subscriptionId, $topicName);
optimistic_subscribe($projectId, $topicName, $subscriptionId);
}
}
# [END pubsub_optimistic_subscribe]
require_once __DIR__ . '/../../../testing/sample_helpers.php';
\Google\Cloud\Samples\execute_sample(__FILE__, __NAMESPACE__, $argv);
2 changes: 2 additions & 0 deletions pubsub/api/src/subscribe_exactly_once_delivery.php
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ function subscribe_exactly_once_delivery(
): void {
$pubsub = new PubSubClient([
'projectId' => $projectId,
// use the apiEndpoint option to set a regional endpoint
'apiEndpoint' => 'us-west1-pubsub.googleapis.com:443'
]);

$subscription = $pubsub->subscription($subscriptionId);
Expand Down
Loading