Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Request for support for HF based models #94

Open
Adamwgoh opened this issue Jan 29, 2024 · 0 comments
Open

Request for support for HF based models #94

Adamwgoh opened this issue Jan 29, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@Adamwgoh
Copy link

I'm currently exploring this library as a first level filter for external model repository, and found that one of the core functionality hasnt been fully supported: the need to scan models saved using huggingface's transformer library.

As far as I understand and tested, models save as .h5 using huggingface's TF library have a different way of storing model configuration compared to tensorflow keras, and thus are not able to be loaded using keras' models.load_model() directly.

Example of how .h5 models saved using transformers:

from transformers import TFAutoModelForImageClassification

tf_model = TFAutoModelForImageClassification.from_pretrained("my_model", from_pt=True)
tf_model.save_pretrained("my_model_tf")

I also noticed that prior to v4.0.0, there were initial support explicitly for huggingface, but opted to move towards a extendable, modular codebase. However, as part of the security evaluation an external open source solution, we want to evaluate if modelscan has intention to support capability to scan most huggingface models over in the long term, and can be adopted.

I'd like to enquire and understand further the team's position on this, and if this is something I can suggest for?

Thanks for your time !

@Adamwgoh Adamwgoh added the enhancement New feature or request label Jan 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant