Skip to content
This repository has been archived by the owner on Mar 19, 2024. It is now read-only.

How is the underlying embedding matrix updated? #1338

Open
SDUxmh opened this issue Jun 9, 2023 · 0 comments
Open

How is the underlying embedding matrix updated? #1338

SDUxmh opened this issue Jun 9, 2023 · 0 comments

Comments

@SDUxmh
Copy link

SDUxmh commented Jun 9, 2023

I realized the fastText code at the underlying , and adopted the gradient ascending method when calculating gradient, which could be completed by training small sample data. However, when training large sample data, update the embedding matrix and add lr*grad to each word vector. After several epoches, the embedding matrix will explode directly (nan). We want to know how the underlying embedding matrix is updated.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant