Skip to content

chore: merge upstream #32

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 133 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
133 commits
Select commit Hold shift + click to select a range
af99b24
Add `Migrated` changelog event
timvaillancourt Jun 4, 2022
ec199f1
revert out of scope change
timvaillancourt Jun 4, 2022
261e1f7
Remove unused code in `go/mysql/binlog.go`
timvaillancourt Jun 6, 2022
eb80144
Remove unused `Type` field
timvaillancourt Jun 6, 2022
bf0c13b
Add table comment to changelog
timvaillancourt Jun 6, 2022
387b325
Var rename
timvaillancourt Jun 6, 2022
de339dd
Fix `integer divide by zero` panic in migrator
timvaillancourt Jun 23, 2022
87970f6
fix(lost data in mysql two-phase commit): lost data in mysql two-phas…
shaohk Jun 24, 2022
38e86e5
Merge branch 'master' into fix-two-phase-commit-lost-data_v2
timvaillancourt Jun 24, 2022
d07c4f0
Run CodeQL analysis on PRs
dm-2 Jul 6, 2022
f0209e8
Merge pull request #1143 from github/codeql-run-on-prs
dm-2 Jul 6, 2022
b80b6e7
Merge branch 'master' into fix-two-phase-commit-lost-data_v2
dm-2 Jul 6, 2022
f35bfd3
Merge branch 'master' into fix-issue-1117
dm-2 Jul 6, 2022
8d9761d
Merge pull request #1141 from shaohk/fix-two-phase-commit-lost-data_v2
dm-2 Jul 6, 2022
b7f7e2c
Merge branch 'master' into fix-issue-1117
dm-2 Jul 6, 2022
3901a86
Merge pull request #1140 from timvaillancourt/fix-issue-1117
dm-2 Jul 6, 2022
cc38a17
compound pk tests (#387)
shlomi-noach Jul 6, 2022
6bf32f2
Ensure mysql rows responses are closed (#1132)
timvaillancourt Jul 6, 2022
0b066c1
Use `switch` statements for readability, simplify `.NewGoMySQLReader(…
timvaillancourt Jul 6, 2022
0918bab
Add context/timeout to HTTP throttle check (#1131)
timvaillancourt Jul 6, 2022
308ba7f
Default to `go1.17.11` (#1136)
timvaillancourt Jul 6, 2022
b751499
Cancel any row count queries before attempting to cut over (#846)
ajm188 Jul 6, 2022
09bb5ca
Merge branch 'master' into go-mysql-binlog-dead-code
timvaillancourt Jul 6, 2022
500fbef
`golang-ci`: fix `staticcheck` linter warnings
timvaillancourt Jul 7, 2022
0ec555b
Cleanup
timvaillancourt Jul 7, 2022
0303031
cleanup v2
timvaillancourt Jul 7, 2022
c3593c9
cleanup v3
timvaillancourt Jul 7, 2022
eead61c
cleanup v4
timvaillancourt Jul 7, 2022
e453570
cleanup v5
timvaillancourt Jul 7, 2022
02258ac
Merge pull request #1137 from github/go-mysql-binlog-dead-code
dm-2 Jul 7, 2022
8eb9708
Merge branch 'master' into changlog-table-comment
dm-2 Jul 7, 2022
6d5d496
Merge branch 'master' into golang-ci-staticcheck-linter
dm-2 Jul 7, 2022
78da0d0
Merge pull request #1138 from github/changlog-table-comment
dm-2 Jul 7, 2022
f9fa423
Merge branch 'master' into golang-ci-staticcheck-linter
dm-2 Jul 7, 2022
eee8ffe
Merge pull request #1145 from timvaillancourt/golang-ci-staticcheck-l…
dm-2 Jul 7, 2022
d6d1d3b
Merge branch 'master' into changelog-migrated-state
timvaillancourt Jul 7, 2022
9c611bd
switch to golang.org/x/term from deprecated golang.org/x/crypto/ssh/t…
dm-2 Jul 7, 2022
246800e
Fix lint
timvaillancourt Jul 7, 2022
54db417
Consolidate no-op states
timvaillancourt Jul 7, 2022
3da9a9c
Merge pull request #1146 from github/switch-to-term-package
dm-2 Jul 7, 2022
607ea8d
Merge branch 'master' into changelog-migrated-state
dm-2 Jul 7, 2022
30c8688
Remove unused `go/os/process.go`
timvaillancourt Jul 7, 2022
582d4cb
Merge branch 'master' into rm-go-os-process-go
timvaillancourt Jul 7, 2022
4648da1
Merge pull request #1134 from github/changelog-migrated-state
dm-2 Jul 8, 2022
ed8c05c
Merge branch 'master' into rm-go-os-process-go
dm-2 Jul 8, 2022
ae02941
Merge pull request #1147 from timvaillancourt/rm-go-os-process-go
dm-2 Jul 8, 2022
f527d63
Move `.Kill()` func from inspector to `go/mysql` (#1148)
timvaillancourt Jul 11, 2022
84dca03
Add script and docs for linter (#1151)
timvaillancourt Jul 18, 2022
ffe54f4
Enable more `golang-ci` linters (#1149)
timvaillancourt Jul 18, 2022
2d3e8d9
Only build RPM and deb packages for amd64
dm-2 Jul 20, 2022
7d8e4e8
Merge pull request #1154 from github/fix-build-script
dm-2 Jul 20, 2022
113094b
Convert character to bytes and insert into table using latin1
wangzihuacool Aug 3, 2022
8a193e0
delete junk files
wangzihuacool Aug 3, 2022
bee009b
Allow zero in dates (#1161)
timvaillancourt Aug 10, 2022
c93060e
restore connection charset to utf8mb4
wangzihuacool Aug 11, 2022
5de91c9
Add missing doc from PR #1131 (#1162)
timvaillancourt Aug 11, 2022
3f44e04
Set a transaction isolation level for MySQL connections (#1156)
timvaillancourt Aug 12, 2022
2ad614a
Merge branch 'master' into fix-charset
timvaillancourt Aug 18, 2022
a17389e
Some fix to unit tests.
wangzihuacool Aug 19, 2022
e9cb4a2
convert to bytes if character string without charsetConversion.
wangzihuacool Aug 27, 2022
ca8aef5
chore: remove duplicate word in comments (#1175)
Abirdcfly Aug 30, 2022
9c2c7ba
Merge branch 'master' into fix-charset
dm-2 Sep 6, 2022
1a473a4
Merge pull request #1158 from wangzihuacool/fix-charset
dm-2 Sep 6, 2022
3c946e9
Improve applier `.ReadMigrationRangeValues()` func accuracy (#1164)
timvaillancourt Sep 6, 2022
1fa3d4f
Add basic tests for applier (#1165)
timvaillancourt Sep 6, 2022
05c7ed5
Add basic test for inspector (#1166)
timvaillancourt Sep 6, 2022
1df37c2
Add basic tests to migrator (#1168)
timvaillancourt Sep 6, 2022
321e584
fix: because lock is not release, drop cutover sentry table is hanged
lmtwga Sep 16, 2022
3f43400
add-rocksdb-as-transactional-engine
wangzihuacool Oct 20, 2022
df4cf7b
Add basic test for hooks (#1179)
timvaillancourt Oct 21, 2022
659e440
Merge branch 'master' into master
dm-2 Oct 21, 2022
9b3fa79
Enable more `golangci-lint` linters (#1181)
timvaillancourt Oct 21, 2022
b538733
Merge branch 'master' into master
dm-2 Oct 21, 2022
2793e2b
Fix: Change table name
hasanMshawrab Oct 26, 2022
515aa72
Print status to migration context logger
SocalNick Oct 27, 2022
1be6a4c
Attempt instant DDL if supported
morgo Nov 10, 2022
05f32eb
minor cleanup
morgo Nov 10, 2022
9cbb42b
fix CI tests to ubuntu-20.04 because ubuntu-22.04 (current -latest) d…
dm-2 Nov 14, 2022
96d708c
Merge pull request #1200 from github/fix-ci
dm-2 Nov 14, 2022
ca12039
Merge branch 'master' into nc--send-status-to-migration-context-logge…
dm-2 Nov 14, 2022
fb82caf
Merge pull request #1194 from SocalNick/nc--send-status-to-migration-…
dm-2 Nov 14, 2022
75a346b
Add tests, incorporate feedback
morgo Nov 14, 2022
b06c1cd
Improve docs
morgo Nov 14, 2022
cfd8372
Merge branch 'master' into attempt-instant-ddl
timvaillancourt Nov 14, 2022
3f3a10a
Address PR feedback
morgo Nov 14, 2022
1b22f78
temp commit to investigate datetime-with-zero test failure
dm-2 Nov 15, 2022
5127519
more testing
dm-2 Nov 15, 2022
74fb8e8
Update go/logic/migrator.go
morgo Nov 15, 2022
5283b46
Make it clear in docs it is disabled by default but safe.
morgo Nov 15, 2022
0310f9f
Merge remote-tracking branch 'origin/attempt-instant-ddl' into attemp…
morgo Nov 15, 2022
59c1a24
remove useless func per review
morgo Nov 15, 2022
a523042
Merge pull request #1201 from morgo/attempt-instant-ddl
dm-2 Nov 16, 2022
a673363
Merge branch 'master' into patch-1
dm-2 Nov 16, 2022
3613f22
Merge pull request #1192 from hasanMshawrab/patch-1
dm-2 Nov 16, 2022
cd65c7e
add extra debugging output
dm-2 Nov 18, 2022
7cebc16
debugging
dm-2 Nov 18, 2022
756f3d3
add error detection for test setup, sort tests to make it easier to t…
dm-2 Nov 18, 2022
8abd584
fix broken test by removing invalid insert statement
dm-2 Nov 18, 2022
45cf63c
Merge branch 'master' into ensure-test-errors-detected
dm-2 Nov 18, 2022
1a131d4
Merge pull request #1202 from github/ensure-test-errors-detected
dm-2 Nov 18, 2022
9c0babd
Merge branch 'github:master' into add-rocksdb-as-transactional-engine
wangzihuacool Nov 23, 2022
f053ccd
support rocksdb as transactional engine
Nov 23, 2022
3ee667a
Modify tests to support rocksdb tests
Nov 25, 2022
ccf5b2e
gofmt
wangzihuacool Nov 25, 2022
9c7857b
SetConnectionConfig
Nov 25, 2022
094909f
Merge branch 'add-rocksdb-as-transactional-engine' of https://github.…
Nov 25, 2022
da35142
add support for rocksdb
Nov 25, 2022
af1e0d6
add support for rocksdb
Nov 25, 2022
ed71099
add percona to versions in workflows
Nov 26, 2022
6e2be1d
add description and optimize tests
Nov 27, 2022
9f3cf74
Apply suggestions from code review
wangzihuacool Nov 29, 2022
20af3af
Apply suggestions from code review
wangzihuacool Nov 29, 2022
7320fda
Merge pull request #1190 from wangzihuacool/add-rocksdb-as-transactio…
dm-2 Nov 29, 2022
bea5323
Merge branch 'master' into master
timvaillancourt Jan 27, 2023
b7db8c6
Merge pull request #1180 from lmtwga/master
rashiq Feb 6, 2023
0a033c7
feat(feat-set-binlogsyncer-maxreconnectattempts): feat-set-binlogsync…
shaohk May 25, 2023
1b3fac2
Add `focal` build
rashiq Sep 19, 2023
00e49a3
Merge pull request #1309 from github/rashiq-focal-build
rashiq Sep 19, 2023
aadc6ab
upgrade codeQL action to v2
meiji163 Sep 25, 2023
671d7a2
Merge pull request #1310 from github/meiji163-update-codeql
meiji163 Sep 25, 2023
e664c1b
Pin docker image to stretch (#1324)
meiji163 Oct 10, 2023
03f7fc8
parse binlog timestamp in UTC (#1322)
meiji163 Oct 10, 2023
94463cf
upgrade standard library deps (#1332)
meiji163 Oct 25, 2023
fe9d8d1
fix: typo (#1331)
testwill Dec 7, 2023
fa12f44
Remove stale RELEASE_VERSION file (#1349)
timvaillancourt Dec 7, 2023
b22229f
Force index in unique key range queries (#1237)
timvaillancourt Dec 7, 2023
d90112c
Include git commit in version (#1347)
timvaillancourt Dec 8, 2023
347d8e0
Fix tokens in inspector logging (#1226)
timvaillancourt Dec 8, 2023
5ebb953
Cleanup whitespace and comments in SQL query text (#1246)
timvaillancourt Dec 8, 2023
679d39e
Fix `--version` output (#1352)
timvaillancourt Dec 11, 2023
462930f
Update to `go1.21` (#1351)
timvaillancourt Dec 11, 2023
8f9c384
Replace deprecated `github.com/satori/go.uuid` lib (#1354)
timvaillancourt Dec 11, 2023
59fd18d
Add `CODEOWNERS` file to auto-request reviews (#1350)
timvaillancourt Dec 12, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
* @dm-2 @rashiq @meiji163 @timvaillancourt
8 changes: 4 additions & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,12 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4

- name: Set up Go 1.16
uses: actions/setup-go@v1
- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: 1.16
go-version-file: go.mod

- name: Build
run: script/cibuild
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ name: "CodeQL analysis"

on:
push:
pull_request:
schedule:
- cron: '0 0 * * 0'

Expand All @@ -18,7 +19,7 @@ jobs:
uses: actions/checkout@v2

- name: Initialize CodeQL
uses: github/codeql-action/init@v1
uses: github/codeql-action/init@v2

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v1
uses: github/codeql-action/analyze@v2
8 changes: 5 additions & 3 deletions .github/workflows/golangci-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,11 @@ jobs:
name: lint
runs-on: ubuntu-latest
steps:
- uses: actions/setup-go@v3
- uses: actions/checkout@v4
- uses: actions/setup-go@v4
with:
go-version: 1.16
- uses: actions/checkout@v3
go-version-file: go.mod
- name: golangci-lint
uses: golangci/golangci-lint-action@v3
with:
version: v1.54.2
12 changes: 6 additions & 6 deletions .github/workflows/replica-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,18 @@ on: [pull_request]
jobs:
build:

runs-on: ubuntu-latest
runs-on: ubuntu-20.04
strategy:
matrix:
version: [mysql-5.7.25,mysql-8.0.16]
version: [mysql-5.7.25,mysql-8.0.16,PerconaServer-8.0.21]

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4

- name: Set up Go 1.16
uses: actions/setup-go@v1
- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: 1.16
go-version-file: go.mod

- name: migration tests
env:
Expand Down
28 changes: 23 additions & 5 deletions .golangci.yml
Original file line number Diff line number Diff line change
@@ -1,12 +1,30 @@
run:
timeout: 5m
modules-download-mode: readonly

linters:
disable:
- errcheck
- staticcheck
enable:
- gosimple
- govet
- bodyclose
- containedctx
- contextcheck
- dogsled
- durationcheck
- errname
- errorlint
- execinquery
- gofmt
- ifshort
- misspell
- nilerr
- nilnil
- noctx
- nolintlint
- nosprintfhostport
- prealloc
- rowserrcheck
- sqlclosecheck
- unconvert
- unparam
- unused
- wastedassign
- whitespace
4 changes: 2 additions & 2 deletions Dockerfile.packaging
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
FROM golang:1.16.4
FROM golang:1.21-bullseye

RUN apt-get update
RUN apt-get install -y ruby ruby-dev rubygems build-essential
RUN gem install --no-ri --no-rdoc fpm
RUN gem install fpm
ENV GOPATH=/tmp/go

RUN apt-get install -y curl
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile.test
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM golang:1.16.4
FROM golang:1.21-bullseye
LABEL maintainer="[email protected]"

RUN apt-get update
Expand Down
1 change: 0 additions & 1 deletion RELEASE_VERSION

This file was deleted.

11 changes: 8 additions & 3 deletions build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,8 @@ function build {

(cd $buildpath && tar cfz ./gh-ost-binary-${osshort}-${GOARCH}-${timestamp}.tar.gz $target)

if [ "$GOOS" == "linux" ] ; then
# build RPM and deb for Linux, x86-64 only
if [ "$GOOS" == "linux" ] && [ "$GOARCH" == "amd64" ] ; then
echo "Creating Distro full packages"
builddir=$(setuptree)
cp $buildpath/$target $builddir/gh-ost/usr/bin
Expand All @@ -51,14 +52,18 @@ main() {
RELEASE_VERSION=$(git describe --abbrev=0 --tags | tr -d 'v')
fi
if [ -z "${RELEASE_VERSION}" ] ; then
RELEASE_VERSION=$(cat RELEASE_VERSION)
echo "RELEASE_VERSION must be set"
exit 1
fi

if [ -z "${GIT_COMMIT}" ]; then
GIT_COMMIT=$(git rev-parse HEAD)
fi

buildpath=/tmp/gh-ost-release
target=gh-ost
timestamp=$(date "+%Y%m%d%H%M%S")
ldflags="-X main.AppVersion=${RELEASE_VERSION}"
ldflags="-X main.AppVersion=${RELEASE_VERSION} -X main.GitCommit=${GIT_COMMIT}"

mkdir -p ${buildpath}
rm -rf ${buildpath:?}/*
Expand Down
4 changes: 2 additions & 2 deletions doc/azure.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@ gh-ost \
--master-user="master-user-name" \
--master-password="master-password" \
--assume-rbr \
[-- other paramters you need]
[-- other parameters you need]
```


[new_issue]: https://github.com/github/gh-ost/issues/new
[assume_rbr_docs]: https://github.com/github/gh-ost/blob/master/doc/command-line-flags.md#assume-rbr
[migrate_test_on_replica_docs]: https://github.com/github/gh-ost/blob/master/doc/cheatsheet.md#c-migratetest-on-replica
[migrate_test_on_replica_docs]: https://github.com/github/gh-ost/blob/master/doc/cheatsheet.md#c-migratetest-on-replica
8 changes: 7 additions & 1 deletion doc/coding-ghost.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
Getting started with gh-ost development is simple!

- First obtain the repository with `git clone` or `go get`.
- From inside of the repository run `script/cibuild`
- From inside of the repository run `script/cibuild`.
- This will bootstrap the environment if needed, format the code, build the code, and then run the unit test.

## CI build workflow
Expand All @@ -14,6 +14,12 @@ Getting started with gh-ost development is simple!

If additional steps are needed, please add them into this workflow so that the workflow remains simple.

## `golang-ci` linter

To enfore best-practices, Pull Requests are automatically linted by [`golang-ci`](https://golangci-lint.run/). The linter config is located at [`.golangci.yml`](https://github.com/github/gh-ost/blob/master/.golangci.yml) and the `golangci-lint` GitHub Action is located at [`.github/workflows/golangci-lint.yml`](https://github.com/github/gh-ost/blob/master/.github/workflows/golangci-lint.yml).

To run the `golang-ci` linters locally _(recommended before push)_, use `script/lint`.

## Notes:

Currently, `script/ensure-go-installed` will install `go` for Mac OS X and Linux. We welcome PR's to add other platforms.
43 changes: 43 additions & 0 deletions doc/command-line-flags.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ A more in-depth discussion of various `gh-ost` command line flags: implementatio

Add this flag when executing on Aliyun RDS.

### allow-zero-in-date

Allows the user to make schema changes that include a zero date or zero in date (e.g. adding a `datetime default '0000-00-00 00:00:00'` column), even if global `sql_mode` on MySQL has `NO_ZERO_IN_DATE,NO_ZERO_DATE`.

### azure

Add this flag when executing on Azure Database for MySQL.
Expand Down Expand Up @@ -41,6 +45,25 @@ If you happen to _know_ your servers use RBR (Row Based Replication, i.e. `binlo
Skipping this step means `gh-ost` would not need the `SUPER` privilege in order to operate.
You may want to use this on Amazon RDS.

### attempt-instant-ddl

MySQL 8.0 supports "instant DDL" for some operations. If an alter statement can be completed with instant DDL, only a metadata change is required internally. Instant operations include:

- Adding a column
- Dropping a column
- Dropping an index
- Extending a varchar column
- Adding a virtual generated column

It is not reliable to parse the `ALTER` statement to determine if it is instant or not. This is because the table might be in an older row format, or have some other incompatibility that is difficult to identify.

`--attempt-instant-ddl` is disabled by default, but the risks of enabling it are relatively minor: `gh-ost` may need to acquire a metadata lock at the start of the operation. This is not a problem for most scenarios, but it could be a problem for users that start the DDL during a period with long running transactions.

`gh-ost` will automatically fallback to the normal DDL process if the attempt to use instant DDL is unsuccessful.

### binlogsyncer-max-reconnect-attempts
`--binlogsyncer-max-reconnect-attempts=0`, the maximum number of attempts to re-establish a broken inspector connection for sync binlog. `0` or `negative number` means infinite retry, default `0`

### conf

`--conf=/path/to/my.cnf`: file where credentials are specified. Should be in (or contain) the following format:
Expand Down Expand Up @@ -226,6 +249,18 @@ Allows `gh-ost` to connect to the MySQL servers using encrypted connections, but

`--ssl-key=/path/to/ssl-key.key`: SSL private key file (in PEM format).

### storage-engine
Default is `innodb`, and `rocksdb` support is currently experimental. InnoDB and RocksDB are both transactional engines, supporting both shared and exclusive row locks.

But RocksDB currently lacks a few features support compared to InnoDB:
- Gap Locks
- Foreign Key
- Generated Columns
- Spatial
- Geometry

When `--storage-engine=rocksdb`, `gh-ost` will make some changes necessary (e.g. sets isolation level to `READ_COMMITTED`) to support RocksDB.

### test-on-replica

Issue the migration on a replica; do not modify data on master. Useful for validating, testing and benchmarking. See [`testing-on-replica`](testing-on-replica.md)
Expand All @@ -242,6 +277,14 @@ Provide a command delimited list of replicas; `gh-ost` will throttle when any of

Provide an HTTP endpoint; `gh-ost` will issue `HEAD` requests on given URL and throttle whenever response status code is not `200`. The URL can be queried and updated dynamically via [interactive commands](interactive-commands.md). Empty URL disables the HTTP check.

### throttle-http-interval-millis

Defaults to 100. Configures the HTTP throttle check interval in milliseconds.

### throttle-http-timeout-millis

Defaults to 1000 (1 second). Configures the HTTP throttler check timeout in milliseconds.

### timestamp-old-table

Makes the _old_ table include a timestamp value. The _old_ table is what the original table is renamed to at the end of a successful migration. For example, if the table is `gh_ost_test`, then the _old_ table would normally be `_gh_ost_test_del`. With `--timestamp-old-table` it would be, for example, `_gh_ost_test_20170221103147_del`.
Expand Down
2 changes: 2 additions & 0 deletions doc/requirements-and-limitations.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ The `SUPER` privilege is required for `STOP SLAVE`, `START SLAVE` operations. Th
- Switching your `binlog_format` to `ROW`, in the case where it is _not_ `ROW` and you explicitly specified `--switch-to-rbr`
- If your replication is already in RBR (`binlog_format=ROW`) you can specify `--assume-rbr` to avoid the `STOP SLAVE/START SLAVE` operations, hence no need for `SUPER`.

- `gh-ost` uses the `REPEATABLE_READ` transaction isolation level for all MySQL connections, regardless of the server default.

- Running `--test-on-replica`: before the cut-over phase, `gh-ost` stops replication so that you can compare the two tables and satisfy that the migration is sound.

### Limitations
Expand Down
2 changes: 1 addition & 1 deletion doc/shared-key.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ CREATE TABLE tbl (

(This is also the definition of the _ghost_ table, except that that table would be called `_tbl_gho`).

In this migration, the _before_ and _after_ versions contain the same unique not-null key (the PRIMARY KEY). To run this migration, `gh-ost` would iterate through the `tbl` table using the primary key, copy rows from `tbl` to the _ghost_ table `_tbl_gho` in primary key order, while also applying the binlog event writes from `tble` onto `_tbl_gho`.
In this migration, the _before_ and _after_ versions contain the same unique not-null key (the PRIMARY KEY). To run this migration, `gh-ost` would iterate through the `tbl` table using the primary key, copy rows from `tbl` to the _ghost_ table `_tbl_gho` in primary key order, while also applying the binlog event writes from `tbl` onto `_tbl_gho`.

The applying of the binlog events is what requires the shared unique key. For example, an `UPDATE` statement to `tbl` translates to a `REPLACE` statement which `gh-ost` applies to `_tbl_gho`. A `REPLACE` statement expects to insert or replace an existing row based on its row's values and the table's unique key constraints. In particular, if inserting that row would result in a unique key violation (e.g., a row with that primary key already exists), it would _replace_ that existing row with the new values.

Expand Down
20 changes: 15 additions & 5 deletions go.mod
Original file line number Diff line number Diff line change
@@ -1,17 +1,27 @@
module github.com/github/gh-ost

go 1.16
go 1.21

require (
github.com/go-ini/ini v1.62.0
github.com/go-mysql-org/go-mysql v1.3.0
github.com/go-sql-driver/mysql v1.6.0
github.com/google/uuid v1.4.0
github.com/openark/golib v0.0.0-20210531070646-355f37940af8
github.com/satori/go.uuid v1.2.0
golang.org/x/net v0.17.0
golang.org/x/term v0.13.0
golang.org/x/text v0.13.0
)

require (
github.com/pingcap/errors v0.11.5-0.20201126102027-b0a155152ca3 // indirect
github.com/satori/go.uuid v1.2.0 // indirect
github.com/shopspring/decimal v0.0.0-20180709203117-cd690d0c9e24 // indirect
github.com/siddontang/go v0.0.0-20180604090527-bdc77568d726 // indirect
github.com/siddontang/go-log v0.0.0-20180807004314-8d05993dda07 // indirect
github.com/smartystreets/goconvey v1.6.4 // indirect
golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83
golang.org/x/net v0.0.0-20210224082022-3d97a244fca7
golang.org/x/text v0.3.6
go.uber.org/atomic v1.7.0 // indirect
golang.org/x/sys v0.13.0 // indirect
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect
gopkg.in/ini.v1 v1.62.0 // indirect
)
21 changes: 10 additions & 11 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ github.com/go-sql-driver/mysql v1.5.0/go.mod h1:DCzpHaOWr8IXmIStZouvnhqoel9Qv2LB
github.com/go-sql-driver/mysql v1.6.0 h1:BCTh4TKNUYmOmMUcQ3IipzF5prigylS7XXjEkfCHuOE=
github.com/go-sql-driver/mysql v1.6.0/go.mod h1:DCzpHaOWr8IXmIStZouvnhqoel9Qv2LBy8hT2VhHyBg=
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/uuid v1.4.0 h1:MtMxsa51/r9yyhkyLsVeVt0B+BGQZzpQiTQ4eHZ8bc4=
github.com/google/uuid v1.4.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1 h1:EGx4pi6eqNxGaHF6qqu48+N2wcFQ5qg5FXgOdqsJ5d8=
github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
github.com/jmoiron/sqlx v1.3.3/go.mod h1:2BljVx/86SuTyjE+aPYlHCTNvZrnJXghYGpNiXLBMCQ=
Expand Down Expand Up @@ -81,33 +83,30 @@ golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACk
golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83 h1:/ZScEX8SfEmUGRHs0gxpqteO5nfNW6axyZbBdw9A12g=
golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20210224082022-3d97a244fca7 h1:OgUuv8lsRpBibGNbSizVwKWlysjaNzmC9gYMhPVfqFM=
golang.org/x/net v0.0.0-20210224082022-3d97a244fca7/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM=
golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68 h1:nxC68pudNYkKU6jWhgrqdreuFiOQWj1Fs7T3VrH4Pjw=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1 h1:v+OssWQX+hTHEmOBgwxdZxK4zHq3yOs8F9J7mk0PY8E=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/sys v0.13.0 h1:Af8nKPmuFypiUBjVoU9V20FiaFXOcuZI21p0ycVYYGE=
golang.org/x/sys v0.13.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.13.0 h1:bb+I9cTfFazGW51MZqBVmZy7+JEJMouUHTUSKVQLBek=
golang.org/x/term v0.13.0/go.mod h1:LTmsnFJwVN6bCy1rVCoS+qHT1HhALEFxKncY3WNNh4U=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6 h1:aRYxNxv6iGQlyVaZmk6ZgYEDa+Jg18DxebPSrd6bg1M=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.13.0 h1:ablQoSUd0tRdKxZewP80B+BaqeKJuVhuRxj/dkrun3k=
golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
Expand Down
Loading