[ENH] Fix 422 Unprocessable Entity error in Chroma Load

[rescrv]

Description of changes

If the bit-difference workload is used for pure-upsert workloads it will
violate quota. This PR shrinks the batch size to just one cluster at a
time. By construction clusters fit a batch.

Test plan

• Tests pass locally with pytest for python, yarn test for js, cargo test for rust

Documentation Changes

N/A

[propel-code-bot]

This PR fixes a quota violation in the bit-difference workload when used for pure-upsert operations. The solution reduces the batch size from multiple clusters to just one cluster at a time, ensuring clusters fit within batch limits.

This summary was automatically generated by @propel-code-bot

[github-actions]

Reviewer Checklist

Please leverage this checklist to ensure your code review is thorough before approving

Testing, Bugs, Errors, Logs, Documentation

• Can you think of any use case in which the code does not behave as intended? Have they been tested?
• Can you think of any inputs or external events that could break the code? Is user input validated and safe? Have they been tested?
• If appropriate, are there adequate property based tests?
• If appropriate, are there adequate unit tests?
• Should any logging, debugging, tracing information be added or removed?
• Are error messages user-friendly?
• Have all documentation changes needed been made?
• Have all non-obvious changes been commented?

System Compatibility

• Are there any potential impacts on other parts of the system or backward compatibility?
• Does this change intersect with any items on our roadmap, and if so, is there a plan for fitting them together?

Quality

• Is this code of a unexpectedly high quality (Readability, Modularity, Intuitiveness)

[propel-code-bot]

[CriticalError]

The loop iteration variable and batch size parameter uq.batch_size have been removed, but there's no check to ensure that enough documents are sampled to meet the original intent. If num_this_cluster is smaller than what would have been generated with multiple clusters in a batch, this could lead to fewer documents being processed than before.

[rescrv]

Thank you. That's my intent.