Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add auxiliary losses directly imposed on params #415

Open
tuffr5 opened this issue Feb 27, 2024 · 1 comment
Open

Add auxiliary losses directly imposed on params #415

tuffr5 opened this issue Feb 27, 2024 · 1 comment

Comments

@tuffr5
Copy link

tuffr5 commented Feb 27, 2024

The general framework for training is:
x --> hash encoding --> network --> y <<loss

If I impose auxiliary losses directly on params like

x --> hash encoding --> network --> y <<loss
           ^               ^
           ^               ^
   auxiliary loss1   auxiliary loss2

I wonder that is it possible in current tiny-cuda-nn? Currently, I tried in the way I indicated above to regularize the params, resulting in a terrible model.

@tuffr5
Copy link
Author

tuffr5 commented Feb 29, 2024

It seems like that the gradient calculation on network is correct, but the hash encoding is not correct. One can do backward multiple times for different losses to get better results, but it slows down the training. Is there any solution for this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant