`drive_service.files().list()` doesn't return trashed items when used with service account JSON

[ErikFub]

When retrieving files from Google Drive using a service account JSON, the API does not return trashed files that actually are in the parents folder(s). This problem does not occur when using default authentication.

The service account has Content Manager rights in the concerned folder. The folder is in a shared drive.

See the below code example for reproduction.

Note: There is a workaround -> ('folder_id' in parents or 'folder_id' in parents)

Environment details

• OS type and version: MacOS Sequoia 15.1.1
• Python version: Python 3.11.9
• pip version: pip 24.3.1
• google-api-python-client version: 2.154.0 2.157.0

Steps to reproduce

from google.oauth2 import service_account
from googleapiclient.discovery import build
import google

# ID of folder to test this with; make sure it contains a trashed item
TEST_FOLDER_ID = 'folder_id'

# Path to your service account key file
SERVICE_ACCOUNT_FILE = 'path_to_sa_json'

def default_credentials():
    credentials, project = google.auth.default()
    return credentials

def service_account_credentials():
    credentials = service_account.Credentials.from_service_account_file(SERVICE_ACCOUNT_FILE)
    return credentials


auth_type_credentials = {
    "default": default_credentials(),
    "service_account": service_account_credentials(),
}

queries = [
    f"'{TEST_FOLDER_ID}' in parents",
    f"('{TEST_FOLDER_ID}' in parents)",
    f"('{TEST_FOLDER_ID}' in parents or '{TEST_FOLDER_ID}' in parents)",
]

for auth_type, credentials in auth_type_credentials.items():
    drive_service = build('drive', 'v3', credentials=credentials)
    print(f"{auth_type=}")
    for query in queries:
        fields = "incompleteSearch, nextPageToken, files(id, name, driveId, mimeType, parents, trashed)"
        results = drive_service.files().list(
            q=query,
            fields=fields,
            supportsAllDrives=True,
            includeItemsFromAllDrives=True,
        ).execute()

        print(f"\t{query=}")
        print(f"\t\t- Cnt results: {len(results['files'])}")
        print(f"\t\t- Of which trashed: {len([f for f in results['files'] if f.get('trashed', False)])}")
    print("\n")

This prints:

auth_type='default'
	query="'folder_id' in parents"
		- Cnt results: 8
		- Of which trashed: 3
	query="('folder_id' in parents)"
		- Cnt results: 8
		- Of which trashed: 3
	query="('folder_id' in parents or 'folder_id' in parents)"
		- Cnt results: 8
		- Of which trashed: 3


auth_type='service_account'
	query="'folder_id' in parents"
		- Cnt results: 5
		- Of which trashed: 0
	query="('folder_id' in parents)"
		- Cnt results: 5
		- Of which trashed: 0
	query="('folder_id' in parents or 'folder_id' in parents)"
		- Cnt results: 8
		- Of which trashed: 3

[ErikFub]

Pushing this as this bug still persists, any help is greatly appreciated :)

[anukaal]

To ensure that trashed files are included in the results when using a service account, you need to explicitly set the includeItemsFromAllDrives and supportsAllDrives parameters to True. Additionally, you can use a query that includes the trashed attribute check.

[ErikFub]

@anukaal Thanks for your comment. If you have a quick look at the code snippet I provided, you can see that all 3 of your suggestions are implemented already. So this unfortunately doesn't really help me.

[anukaal]

it seems that the service account may have limitations or different behavior when querying trashed items

[ErikFub]

Yes, that's what this whole issue is about 🙃