Description
Describe the Bug
when the source of migration scripts is a remote github repository, and one or more of the migration files are >=1MB in size, go migrate fails with error: unsupported content encoding: none
Steps to Reproduce
Steps to reproduce the behavior:
to help reproduce this issue, I have created an example setup where you can launch a postgres container, and run the example migration scripts which trigger this issue, by following the README - https://github.com/georgewheatcroft/go-migrate-issue-large-github-source-files/blob/master/README.md
Expected Behavior
A clear and concise description of what you expected to happen.
go-migrate fails an errors when running (or rather, attempting to download) the large (>1MB) script during the migration run, with:
error: unsupported content encoding: none
Migrate Version
v4.15.2
Loaded Source Drivers
Source drivers: gitlab, go-bindata, godoc-vfs, gcs, s3, bitbucket, github, github-ee, file
Loaded Database Drivers
Database drivers: cockroach, firebirdsql, mongodb, neo4j, cassandra, crdb-postgres, mysql, redshift, spanner, mongodb+srv, pgx, postgresql, sqlserver, clickhouse, cockroachdb, firebird, postgres, stub
Go Version
go version go1.19.5 darwin/arm64
Stacktrace
...
...
2023/03/21 21:09:23 Read and execute 1/u create_car_emissions_table
2023/03/21 21:09:23 Finished 1/u create_car_emissions_table (read 14.076442875s, ran 13.365167ms)
2023/03/21 21:09:23 error: unsupported content encoding: none
Additional context
as detailed in the example repository I have created (includes api call examples) - it
seems that there is a difference in what is returned by the github api depending on file size.
the go-migrate tool github source expects that returned filed content should have all the required file content in the response from the go-github function 'GetContents' - however the api this calls does not have the same behaviour when files are >1MB in size, and does not return the expected response data (i.e. content encoding is '' and there is no direct content data in the response returned)