Skip to content

预训练的时候在加载模型的过程中卡住 #917

Open
@amanyara

Description

@amanyara

Describe the bug
A clear and concise description of what the bug is.
在我预训练模型的时候,模型加载的一晚上还在加载。显存没有跑满。

To Reproduce
Steps to reproduce the behavior:

  1. cd /home/pengc/ernie/ernie-develop/demo
  2. python pretrain.py --data_dir "../data/*.gz" --from_pretrained ../ernie-gram-zh --save_dir ./outabtrain_1012

Expected behavior
A clear and concise description of what you expected to happen.
期待模型开始训练,日志输出内容
054A01C8-4E38-4b0c-860F-060009F1ABB6

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.
运行环境:Ubuntu18.04
显卡:A800*2
Python:3.8.18
cuDNN:8.4
PaddlePaddle-GPU: 2.5.1.post120
运行指令:如图

Metadata

Metadata

Assignees

No one assigned

    Labels

    wontfixThis will not be worked on

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions