Skip to content

redis8分片集群至8分片集群,插槽数报错 #954

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mhn254 opened this issue Apr 30, 2025 · 8 comments
Open

redis8分片集群至8分片集群,插槽数报错 #954

mhn254 opened this issue Apr 30, 2025 · 8 comments
Labels
type: question Further information is requested

Comments

@mhn254
Copy link

mhn254 commented Apr 30, 2025

问题描述(Issue Description)

redis8分片集群至8分片集群,插槽数报错
请在这里简要描述你遇到的问题。
迁移时遇到集群插槽数报错
Please provide a brief description of the issue you encountered.

环境信息(Environment)

  • RedisShake 版本(redis-shake-v4.4.0-linux-arm64):
  • Redis 源端版本(5.0):
  • Redis 目的端版本(5.0):
  • Redis 部署方式(cluster):
  • 是否在云服务商实例上部署( Cloud Provider):

日志信息(Logs)

如果有错误日志或其他相关日志,请在这里提供。

2837 myself,master - 0 1745983196000 7 connected 0-16383
d34fe71a69f4af2e5571de78cb6e682b95bc8f85 172.16.48.13:6379@14276 master - 0 1745983202000 4 connected 0-16383
2a773ec0109eefb5d0b08c2cb6194f2384565a7c 172.16.48.13:6379@12097 master - 0 1745983202000 13 connected 0-16383
be9e94a2cd07a7a5fcb7f9f405397820b8b9b0bf 172.16.48.13:6379@13110 master - 0 1745983200543 15 connected 0-16383
d510f6cc49d4a40eb97c2ad8e46491252b95ffef 172.16.48.13:6379@13975 master - 0 1745983199000 9 connected 0-16383
b3f01b79c3f657d6bda00b814f59301872cf84e5 172.16.48.13:6379@14314 master - 0 1745983201000 6 connected 0-16383
00c44e747740f98a9d05e7891a7db271c94ba316 172.16.48.13:6379@14087 master - 0 1745983199743 3 connected 0-16383
07e6a39dbab2efc30219521de1bf58da4d96a059 172.16.48.13:6379@14183 master - 0 1745983198739 5 connected 0-16383
346e1d2ccc652848f5f1753c67b6317b738e71ea 172.16.48.13:6379@13344 slave 07e6a39dbab2efc30219521de1bf58da4d96a059 0 1745983201545 5 connected
a317f7f8ed010d748c47bb07100b957df3befef3 172.16.48.13:6379@12651 slave be9e94a2cd07a7a5fcb7f9f405397820b8b9b0bf 0 1745983202749 15 connected
047339f1882d3793575159df510c5e8b4028bd6b 172.16.48.13:6379@12848 slave d34fe71a69f4af2e5571de78cb6e682b95bc8f85 0 1745983197000 11 connected
c63b28f75a1ec77bd29fc7fd677ecf6fad98cfaa 172.16.48.13:6379@13664 slave b3f01b79c3f657d6bda00b814f59301872cf84e5 0 1745983198000 12 connected
f619e9657669b5dbd12856aa8790e8910324ae1a 172.16.48.13:6379@13944 slave 00c44e747740f98a9d05e7891a7db271c94ba316 0 1745983200743 3 connected
4ec34be955f8934138b5671178964dcf0261d17a 172.16.48.13:6379@11531 slave cd3c80e1f2be63d4aedf6ed6be8d0c798beda0a8 0 1745983200000 7 connected
5449a2fe0503cbfbad4e05917d63bab97c04ba08 172.16.48.13:6379@12741 slave 2a773ec0109eefb5d0b08c2cb6194f2384565a7c 0 1745983202000 13 connected
c2352d32be634bd3644bf7e8ef5d864ee0d419af 172.16.48.13:6379@12409 slave d510f6cc49d4a40eb97c2ad8e46491252b95ffef 0 1745983197000 10 connected
2025-04-30 11:20:03 ERR invalid cluster nodes slots. slots_count=131072, address=****:6379
RedisShake/internal/utils/cluster_nodes.go:100 -> GetRedisClusterNodes()
RedisShake/internal/reader/scan_cluster_reader.go:17 -> NewScanClusterReader()
RedisShake/cmd/redis-shake/main.go:109 -> main()
runtime/proc.go:267 -> main()
runtime/asm_amd64.s:1650 -> goexit()

If there are any error logs or other relevant logs, please provide them here.

其他信息(Additional Information)

请提供任何其他相关的信息,如配置文件、错误信息或截图等。

Please provide any additional information, such as configuration files, error messages, or screenshots.

@mhn254 mhn254 added the type: question Further information is requested label Apr 30, 2025
@suxb201
Copy link
Member

suxb201 commented May 9, 2025

我去,这是哪家云厂商的集群?

@mhn254
Copy link
Author

mhn254 commented May 9, 2025

我去,这是哪家云厂商的供应?

腾讯

@suxb201
Copy link
Member

suxb201 commented May 9, 2025

源目都是腾讯吗,具体什么产品,是否有代理节点。可以跑下 cluster nodes 看下吗?

@mhn254
Copy link
Author

mhn254 commented May 9, 2025

源目标都是腾讯吗,具体什么产品,是否有代理节点。可以跑下集群节点看下吗?

云数据库Redis@,5.0版本,8分片,自带proxy,只提供了一个vip地址连接,没有直连节点的信息。只要把cluster设置成true就会报这个错,cluster设置成false,程序跑起来随机分配到一个分片,其他分片就同步不了

@suxb201
Copy link
Member

suxb201 commented May 9, 2025

cluster设置成false,程序跑起来随机分配到一个分片,其他分片就同步不了

感觉不应该呢,你直接 redis-cli 连上去 dbsize 看到的数量是全部 db 的吗?

是的话,可以试试用 scanReader 模式,然后 cluster false 去搞。

@mhn254
Copy link
Author

mhn254 commented May 9, 2025

cluster设置成false,程序跑起来随机分配到一个分片,其他分片就同步不了

感觉不应该呢,你直接redis-cli连上去dbsize看到的数量是全部db的吗?

是的话,可以尝试用 scanReader 模式,然后 cluster false 去搞。

源库不支持PSync,所以一直都是用的scanReader搞的, cluster false 启动以后就会出现刚刚说的情况,比如总数据10条落在不用分片上,程序每次启动,read_count的数量都不一样,都是显示其中一个分片的数据,相应的,源库新增数据如果不是写在程序显示的当前分片,就无法增量同步。主备版本一切正常,全量+增量都可以,就集群版不行,目前冷备下来了,反馈这个问题是希望咱能考虑到云上的魔改redis,以后适配度更广。

@suxb201
Copy link
Member

suxb201 commented May 9, 2025

不能理解,cluster false 启动以后如果 RedisShake 上去只能拉到一个分片的数据,你们正常应用应该也只能访问的一个分片的数据。那岂不是正常业务都有问题了。

@mhn254
Copy link
Author

mhn254 commented May 9, 2025

不能理解,cluster false启动以后如果RedisShake上去只能拉到一个分片的数据,你们正常的应用应该也只能访问到一个分片的数据。那不是正常业务都有问题了。

我也不清楚腾讯的redis是个什么情况,不过业务连接到它提供的vip后,是正常的,cluster false启动以后RedisShake的数据就不全,我用测试库插入10条数据,程序启动10次,可能有8次都是读到0条数据的,大佬有兴趣,以后可以去腾讯云模拟一下,我也不是很懂,哈哈下班

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants