Skip to content

关于RWKV6的代码什么时候可以公开? #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
chenzean opened this issue Jul 21, 2024 · 4 comments
Open

关于RWKV6的代码什么时候可以公开? #1

chenzean opened this issue Jul 21, 2024 · 4 comments

Comments

@chenzean
Copy link

No description provided.

@Yaziwel
Copy link
Owner

Yaziwel commented Jul 22, 2024

你好,我还在调试,Vision-RWKV把V6改成bidirectional attention了,你可以借鉴一下:https://github.com/OpenGVLab/Vision-RWKV/tree/master/classification/mmcls_custom/models/backbones

@chenzean
Copy link
Author

好滴,期待作者你的代码

@chenzean
Copy link
Author

想问一下作者,我还有几个问题:

  1. 为什么需要把序列长度限制在512 * 512?RWKV不是说更加适合长序列吗?

  2. wkv_cuda他是c++的代码,能否查看其具体的操作?

@Yaziwel
Copy link
Owner

Yaziwel commented Jul 22, 2024

想问一下作者,我还有几个问题:

  1. 为什么需要把序列长度限制在512 * 512?RWKV不是说更加适合长序列吗?
  2. wkv_cuda他是c++的代码,能否查看其具体的操作?

1,因为我的图像最高分辨率是这么大,所以我设置的512×512,你可以按照你的需求去设置。
2,不太懂查看具体操作是啥意思,意思是设置断点查看每一步吗?这个你可以去问一下Vision-RWKV那边

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants