Feature Request: Add Bulk Delete Operation for Specific Keys in Go Client (clientv3)

[BharathMC]

What would you like to be added?

Hi Team,

This is in context of performance issue mentioned here : #16287. We are looking for a new method to do bulk delete by list of keys. Not by prefix, because when the keys are created with UUID, prefix does not work.

Current behaviour:

The etcd/clientv3 Go library currently lacks a dedicated, efficient method to delete multiple specific (potentially non-contiguous) keys in a single API call. Existing options have drawbacks for certain high-throughput use cases:

1. Individual Delete Calls: Requires N separate RPC calls to delete N keys, which is inefficient due to network latency and server overhead, especially for large N.
2. Delete with WithPrefix() or WithRange(): Effective for deleting keys within a known prefix or range, but not suitable for deleting an arbitrary list of specific keys scattered across the keyspace. Or when we create the keys based on UUIDs
3. Using Txn: Multiple OpDelete operations can be bundled into a transaction. However, this can be cumbersome to construct for large lists of keys and may run into transaction operation limits or request size limits depending on the number of keys.

Desired behaviour with new feature request

Introduce a new function in the clientv3.KV interface that allows deleting a list of specific keys in a single, optimized RPC call. For example, an API signature like:

DeleteBatch(ctx context.Context, keys []string, opts ...OpOption) (*DeleteResponse, error)

Why is this needed?

Use Case
Our application requires inserting approximately 2000 unique keys per second into etcd, each with a short intended lifespan of about 60 seconds, necessitating reliable deletion afterwards. While etcd's lease feature provides TTL functionality, the high churn rate—creating 2000 new leases per second and potentially managing up to 120,000 active leases simultaneously—raises significant concerns about the performance overhead and potential strain on the etcd cluster's lease management system.

Proposed Solution
The client library could potentially implement DeleteBatch by internally constructing a Txn request containing multiple OpDelete operations for the provided list of keys.

[serathius]

Txn: Multiple OpDelete operations can be bundled into a transaction. However, this can be cumbersome to construct for large lists of keys and may run into transaction operation limits or request size limits depending on the number of keys.

Don't understand, can you clarify what is cumbersome?

Our application requires inserting approximately 2000 unique keys per second into etcd, each with a short intended lifespan of about 60 seconds, necessitating reliable deletion afterwards.

What you mean by reliable? To you want to guarantee deletion within 60s? Leases are not good for that as change of leader will refresh the lease. On the other hand if you delete them manually, how do you handle your application crashing?

While etcd's lease feature provides TTL functionality, the high churn rate—creating 2000 new leases per second and potentially managing up to 120,000 active leases simultaneously

You can batch leases and attach single lease to multiple keys, this is what K8s does.