fix uncaught init of linear layer in clip's/siglip's for image classification models

[vasqu]

What does this PR do?

Fixes the uncaught initialisation of the last linear layer in ForImageClassification of Clip/Siglip, especially when using from_pretrained. More in-depth discussion at #30374. Possible failing tests are equivalence tests between tf/pt/flax; seems unrelated.

The remaining discrepancy between from_pretrained and "from_config" has yet to be resolved. I'll leave it out since it might imply bigger things than reflected here.

Fixes #30374

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@amyeroberts @Xmaster6y

[amyeroberts]

Thanks for fixing so quickly!

[Xmaster6y]

LGTM!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[amyeroberts]

@vasqu Could you push an empty commit to this branch? This should trigger the slow model tests to make sure everything's still working as expected for CLIP

[vasqu]

@amyeroberts pushed an empty commit, but seems like the slow test still hasn't been triggered.

[vasqu]

Oh might've been that it's just slow 😅 my bad

[amyeroberts]

@vasqu Slow by name, slow by nature ;) I think it was actually waiting for me to approve - it won't run automatically for non-repo maintainers for security and to save our CI as they're quite heavy tests

[vasqu]

Ah, I see. Makes sense and yea, the name should've rang a bell ^^

[amyeroberts]

Thanks again for fixing this!