Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize Baby LLaMA for Duo #2

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

forcekeng
Copy link

队长:耿力

@AliveGh0st
Copy link

不知道为什么,我在我的duo上直接使用了你的runq_best二进制文件进行测试,速度并未有显著区别,平均大约1.1 token/s ,达不到你表中所测试的6 token/s,和直接使用-Ofast编译 + int8量化后的模型速度基本一致,我的镜像是直接从官方下载的,内存大小为28.5mb,你的README中我看到截图使用的是修改过的55mb内存的镜像,可能是这个原因达不到6 token/s 。
如果是这样的话,那么可能 编译选项和使用RVV加速对于速度的影响似乎微乎其微,在cpu不变的前提下,内存大小是影响速度的最主要因素。

@bekcpear
Copy link

@forcekeng

尊敬的参赛选手,您好。

本次锦标赛您所提交的 PR 初步验证结果已公示于:
https://github.com/plctlab/rvspoc/tree/main/Results/Verifications/S2311

请查阅后进行确认,如有任何异议请回复本条评论。如确认无误请回复「确认无误」,感谢您的配合。

此确认环节为期一周,详情可见 https://rvspoc.org/03/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants