Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent crash with WandbCallback with third parties #30477

Merged
merged 3 commits into from
Apr 25, 2024

Conversation

tomaarsen
Copy link
Member

What does this PR do?

This PR fixes a dodgy if-statement introduced in #30135 that attempts to check whether a model has a num_parameters method. Currently, if

(is_torch_available() and isinstance(model, torch.nn.Module))

is True, then the model does not necessarily have a num_parameters method. Instead, I've replaced it with an "Easier to ask for forgiveness than permission" principle which should be much safer.

This is currently breaking Sentence Transformers training with W&B.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?

Who can review?

@amyeroberts as she reviewed #30135
@muellerz

  • Tom Aarsen

Sorry, something went wrong.

Copy link
Collaborator

@amyeroberts amyeroberts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing!

self._wandb.config["model/num_parameters"] = model.num_parameters()
except AttributeError:
pass
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add a logger.info here (feel free to change the message)? By default in transformers info level isn't logged, but might be useful if someone goes into debug mode

Suggested change
pass
logger.info("Unable to get num_parameters from model")

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call, resolved via a6d09f5

@amyeroberts
Copy link
Collaborator

@tomaarsen Let me know if you need me to merge this in once CI is all green

@tomaarsen
Copy link
Member Author

@amyeroberts I forget what the current merging protocol is for transformers. Do only the core maintainers merge?
I'm good with either way :)

  • Tom Aarsen

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@amyeroberts
Copy link
Collaborator

@tomaarsen Once a core maintainer has approved, anyone is free to merge, I just forget who has permission to

@tomaarsen
Copy link
Member Author

I see! Makes sense :) I'll merge it.

  • Tom Aarsen

@tomaarsen tomaarsen merged commit ce5ae5a into huggingface:main Apr 25, 2024
21 checks passed
@tomaarsen tomaarsen deleted the wandb_third_party branch April 25, 2024 10:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants