Skip to content

Add startIndex to bigquery.table.Table.fetch_data #3081

Closed
@kylemcdonald

Description

@kylemcdonald

The API describes a startIndex parameter for the "Tabledata: list" endpoint. But the implementation doesn't include this parameter. An example of a class does implement a startIndex parameter is bigquery.query.Query as described in the "Jobs: getQueryResults" endpoint.

I tried editing the code to submit a PR, but I'm not familiar with testing this kind of Python package locally (pip install -e . didn't install the packages from their local copy as I expected). In lieu of a PR I can share this patched class TableWithStartIndex that works for my use case:

from google.cloud.bigquery.table import Table, _item_to_row, _rows_page_start
from google.cloud.iterator import HTTPIterator
class TableWithStartIndex(Table):
    def fetch_data_with_start_index(self, max_results=None, page_token=None, client=None, start_index=None):
        client = self._require_client(client)
        path = '%s/data' % (self.path,)
        extra_params = {}
        if start_index is not None:
            extra_params['startIndex'] = start_index
        iterator = HTTPIterator(client=client, path=path,
                                item_to_value=_item_to_row, items_key='rows',
                                page_token=page_token, max_results=max_results,
                                page_start=_rows_page_start,
                                extra_params=extra_params)
        iterator.schema = self._schema
        # Over-ride the key used to retrieve the next page token.
        iterator._NEXT_TOKEN = 'pageToken'
        return iterator

I added three small chunks: the start_index=None argument, this block:

        extra_params = {}
        if start_index is not None:
            extra_params['startIndex'] = start_index

And the bit that passes the extra_params to the HTTPIterator: extra_params=extra_params.

Metadata

Metadata

Assignees

Labels

api: bigqueryIssues related to the BigQuery API.type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions