[BUG] [PYTHON] Datetime params not encoded for request when defined as a list

[esthervogt]

Bug Report Checklist

• Have you provided a full/minimal spec to reproduce the issue?
• Have you validated the input using an OpenAPI validator (example)?
• Have you tested with the latest master to confirm the issue still exists?
• Have you searched for related issues/PRs?
• What's the actual output vs expected output?
• [Optional] Sponsorship to speed up the bug fix or feature request (example)

Description

When adding request parameters to an endpoint that are defined to be datetime strings, they will not be encoded using urllib.parse.quote() if they are grouped as a list, e.g.:

"parameters": [
  {
    "name": "selldate",
    "in": "query",
    "schema": {
      "type": "array",
      "items": {
        "type": "string",
        "format": "date-time"
      }
    }
  }
],

In contrast, the encoding is performed when the parameter is a single datetime string. I would expect to have datetime strings be handled the same way no matter whether they are added as a parameter list or not (see exemplary output in section below: "Suggest a fix").

The reason for this is that for list params, the generator defines a collection_format which has the value 'multi'. No collection format is set for single datetime strings. In the method ApiClient.parameters_to_url_query(), params of type 'multi' are added to the list of params without further encoding in contrast to those without a collection_format:

if k in collection_formats:
    collection_format = collection_formats[k]
    if collection_format == "multi":
        new_params.extend((k, value) for value in v)
...
else:
    new_params.append((k, quote(str(v))))

This leads to an error response if timezone aware datetime strings are used since the timezone offset ('+01:00) is not encoded properly. The missing encoding doesn't seem to be an issue w.r.t. the ':'. Therefore, the autogenerated client only allows to send a list of timezone unaware datetime strings.

These are the url parameters created by PetsApi._list_pets_serialize() that show the differences between the encoded datetimes of birthdate and selldate:

/pets?selldate=2023-01-01T00:00:00+01:00&selldate=2023-02-01T00:00:00+01:00&selldate=2023-03-01T00:00:00+01:00&birthdate=%28%272020-01-01T00%3A00%3A00%2B01%3A00%27%2C%29

openapi-generator version

v7.2.0

OpenAPI declaration file content or url

Gist with minimal petstore example (JSON) can be found here

Generation Details

docker run --rm -v ${PWD}:/local openapitools/openapi-generator-cli:v7.2.0 generate \
-i /local/specs/petstore_min_example.json \
-g python \
-o /local \
--additional-properties="generateSourceCodeOnly=true,packageName=client.openapi_client,library=urllib3"

Steps to reproduce

1. Use the OpenApi spec provided here to auto-generate a client
2. Inspect the outputs of PetsApi._list_pets_serialize(), e.g. by using the snippet provided here

Related issues/PRs

No similar issue found

Suggest a fix

The encoding of datetimes in a list of params is performed if the value in _collection_formats is set to 'csv' instead of 'multi'. The _collection_formats are defined in PetsApi._list_pets_serialize():

_collection_formats: Dict[str, str] = {
    "selldate": "multi",
}

When manually adjusting the _collection_formats as suggested after auto-generating the client, the exemplary result given above looks like this:

/pets?selldate=2023-01-01T00%3A00%3A00%2B01%3A00,2023-02-01T00%3A00%3A00%2B01%3A00,2023-03-01T00%3A00%3A00%2B01%3A00&birthdate=%28%272020-01-01T00%3A00%3A00%2B01%3A00%27%2C%29

[wing328]

FYI. yesterday we merged a fix for list of integers in the query parameters: #17638

maybe we can file a similar fix but for list of datetime query parameters?

[esthervogt]

Hi @wing328, sorry for the delayed response...wasn't sure whether this question was addressed to me. As far as I understand the solution in #17638, I agree that sth similar might be the way to go for a list of datetime params 👍

[wing328]

have you tested the latest master?

link to snapshot version can be found in the project's readme.

[wing328]

FYI. the fix in #176348 convert everything to string using str() so it may work for your use case (list of datetime params) as well.

[esthervogt]

Will try again and give you an update soon

[esthervogt]

I tested with the latest snapshot jar (7.3.0). However, the conversion to strings does not change the encoding. Imo, this would have to be applied earlier, e.g. in the condition of parameters_to_url_query that applies formatting to collection_formats of type 'multi':

            if k in collection_formats:
                collection_format = collection_formats[k]
                if collection_format == 'multi':
                    new_params.extend((k, str(value)) for value in v)
                else:
                    if collection_format == 'ssv':
                        delimiter = ' '
                    elif collection_format == 'tsv':
                        delimiter = '\t'
                    elif collection_format == 'pipes':
                        delimiter = '|'
                    else:  # csv is the default
                        delimiter = ','
                    new_params.append(
                        (k, delimiter.join(quote(str(value)) for value in v))
                    )
            else:
                new_params.append((k, quote(str(v))))

        return "&".join(["=".join(map(str, item)) for item in new_params])

I think, changing new_params.extend((k, str(value)) for value in v) to new_params.extend((k, quote(str(value))) for value in v) might do the trick.

[wing328]

can you please file a PR to start with? we will review accordingly