Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] 使用镜像创建虚拟机时一直在缓存镜像中,超过2个小时就直接失败? #19958

Closed
xxhacker opened this issue Apr 12, 2024 · 9 comments
Labels

Comments

@xxhacker
Copy link

xxhacker commented Apr 12, 2024

我有一个虚拟机使用的镜像是从别的地方导入过来的大概1T左右,在创建的时候不管是系统盘还是数据盘都我都指定使用ceph,但一直卡在了缓存镜像中,我观察有几下现象如图:
1712895557168
不知道上面的缓存列表有什么作用?
1712895628665(1)
我明明指定创建虚拟机使用的空间在是ceph,不知道为何跑到了某一台计算节点上了(好像是随机指定的一台),且这个文件一直增加,感觉是缓存到这里来了

@xxhacker xxhacker added the question Further information is requested label Apr 12, 2024
@hoganlxj
Copy link
Contributor

@xxhacker 用新镜像第一次新建虚拟机,是先把镜像缓存宿主机上,后面再新建虚拟机就会用镜像存储。 1T的镜像镜像,上传到宿主机比较慢,可以再等等

@xxhacker
Copy link
Author

xxhacker commented Apr 12, 2024

@hoganlxj 今天上传了一天了,还是没有上传好,好像这个缓存到40%左右的时候,也就是这个tmp缓存临时文件到540G的时候,就失败了,我看一下系统磁盘占用率为66%左右,还有差不多300多G的空间,报错日志如下:
{
"reason": {
"reason": {
"reason": {
"reason": "{"reason":{"reason":"{\"reason\":{\"reason\":{\"image_id\":\"55e825c5-5cc2-4007-89e3-ad651802ba56\",\"reason\":{\"reason\":\"AcquireImage: LocalStorage.AcquireImage: remoteFile.Fetch: download: io.Copy to tmpPath /opt/cloud/workspace/disks/image_cache/55e825c5-5cc2-4007-89e3-ad651802ba56.tmp from reader: unexpected EOF\",\"stage\":\"OnImageCacheComplete\",\"status\":\"error\"}},\"stage\":\"OnImageCacheComplete\"},\"stage\":\"OnStorageCacheImageComplete\",\"status\":\"error\",\"task_name\":\"StorageCacheImageTask\"}","stage":"OnStorageCacheImageComplete"},"stage":"on_kvm_disk_prepared","status":"error","task_name":"DiskCreateTask"}",
"stage": "on_kvm_disk_prepared"
},
"stage": "on_disk_prepared",
"status": "error",
"task_name": "KVMGuestCreateDiskTask"
},
"stage": "on_disk_prepared"
},
"stage": "OnDiskPrepared",
"status": "error",
"task_name": "GuestCreateDiskTask"
}

@xxhacker
Copy link
Author

能过几次查看我发现,如果缓存时候超过2个小时就失败了,是不是有什么超时设置?
Quicker_20240413_091610
Snipaste_2024-04-13_09-17-57
Snipaste_2024-04-13_09-19-33

@xxhacker xxhacker changed the title [求助/Help] 创建虚拟机是一直在缓存镜像中 [求助/Help] 创建虚拟机是一直在缓存镜像中,超过2个小时就直接失败? Apr 13, 2024
@xxhacker xxhacker changed the title [求助/Help] 创建虚拟机是一直在缓存镜像中,超过2个小时就直接失败? [求助/Help] 使用镜像创建虚拟机时一直在缓存镜像中,超过2个小时就直接失败? Apr 13, 2024
@xxhacker xxhacker changed the title [求助/Help] 使用镜像创建虚拟机时一直在缓存镜像中,超过2个小时就直接失败? [BUG] 使用镜像创建虚拟机时一直在缓存镜像中,超过2个小时就直接失败? Apr 13, 2024
@ioito
Copy link
Collaborator

ioito commented Apr 15, 2024

#19976 按这个改下,更新下glance服务

@xxhacker
Copy link
Author

xxhacker commented Apr 15, 2024

@ioito 按照#19976 代码更新了一下glance服务,把超时时间设置为8个小时,我看日志显示镜像缓存已经100%了,但查看虚拟机的状态还是在缓存镜像中且进度条一动也不动了好几个小时了,一直是83.33%,不知道什么原因,这个/opt/cloud/workspace/disks/image_cache目录下的文件还是tmp,也没有日志报错或者不知道在哪里看日志了
Snipaste_2024-04-15_23-18-17
Snipaste_2024-04-15_23-19-20
Snipaste_2024-04-15_23-39-02

@ioito
Copy link
Collaborator

ioito commented Apr 16, 2024

是否创建时选了共享存储,例如ceph,如果是,83.33%之后需要把镜像导入到ceph中

@xxhacker
Copy link
Author

@ioito 后端是ceph,就是这个镜像比较大缓存比较久,现在的状态是在分配磁盘中,由于这个镜像比较大,所以分配磁盘时间也相当久,不知道能在哪里可以查看分配磁盘的进度,已经分配了快24小时了

@ioito
Copy link
Collaborator

ioito commented Apr 17, 2024

目前没有地方展示导入镜像到ceph的进度

Copy link

If you do not provide feedback for more than 37 days, we will close the issue and you can either reopen it or submit a new issue.

您超过 37 天未反馈信息,我们将关闭该 issue,如有需求您可以重新打开或者提交新的 issue。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants