Skip to content

fix: fix rows returned when both start_index and page_size are provided #2181

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

Linchin
Copy link
Contributor

@Linchin Linchin commented May 13, 2025

TODO:

  • move deep copy import
  • double check total rows - done, total rows should not be affected by start_index
  • unit and system tests

Fixes #2173 🦕

@product-auto-label product-auto-label bot added size: s Pull request size is small. api: bigquery Issues related to the googleapis/python-bigquery API. labels May 13, 2025
@product-auto-label product-auto-label bot added size: m Pull request size is medium. and removed size: s Pull request size is small. labels May 27, 2025
@Linchin Linchin marked this pull request as ready for review May 27, 2025 23:02
@Linchin Linchin requested review from a team as code owners May 27, 2025 23:02
@Linchin Linchin requested a review from logachev May 27, 2025 23:02
@Linchin Linchin assigned chalmerlowe and unassigned shollyman May 28, 2025
{"f": [{"v": "pqe"}]},
],
}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please go through the rest of this test and provide some additional comments.

Clarify why we are using total_rows of 7.
Clarify why setting total_rows to 7 despite only have six rows of tabledata is ok/reasonable
Clarify what is at the indexed positions:

  • tabledata_list_request_1[1]
  • tabledata_list_request_2[1]

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clarify why we are using total_rows of 7.
Clarify why setting total_rows to 7 despite only have six rows of tabledata is ok/reasonable

Done.

Clarify what is at the indexed positions:
tabledata_list_request_1[1]
tabledata_list_request_2[1]

There are comments above the code block explaining that these are the first and the second requests that the client sends to the server.

Comment on lines +1688 to +1689
# request. However, in the subsequent requests, we will pass only
# page_token but not start_index, because the server only allows one
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we mention sending page_token, but nowhere in the test do we confirm that page_token is set or that it has been sent, etc.

In fact, while it is correct, with this language in place, it seems weird that in tabledata_resource_2, pagetoken is set to None. The above language implies that we might send something besides None.

Should we revise the language slightly?
Do we mean some other argument/attribute besides page_token?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the last message, the page token will be empty because there is no following page.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, tabledata_resource_2 is what the server returns, not what we pass to the server. The process of sending the page token happens within result = job.result(page_size=page_size, start_index=start_index) and I don't think it's realistic to expose that in unit test.

Copy link
Collaborator

@chalmerlowe chalmerlowe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some comments about the docstrings and comments in the test.
The test feels complicated especially upon first read so it might benefit from a bit more commentary and potentially some rewording.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. size: m Pull request size is medium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

QueryJob.result() - pages don't respect the start index
4 participants