fix(gemma4_unified): conversion script and config bugs#46398
Conversation
ArthurZucker
left a comment
ArthurZucker
left a comment
There was a problem hiding this comment.
Nothing really actionnable (no patch needed as config is mostly on the hub and conversion scripts are not shipped part of the release)
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| head_dim: int = 256 | ||
| hidden_activation: str = "gelu_pytorch_tanh" | ||
| max_position_embeddings: int = 131_072 | ||
| max_position_embeddings: int = 262_144 |
There was a problem hiding this comment.
You will need to modify the modular (that's why the CI is failing)
On the hub we still have 131_072, https://huggingface.co/google/gemma-4-12B-it/blob/main/config.json#L109 - should this be updated?
There was a problem hiding this comment.
There was a problem hiding this comment.
modular now updated to match.
|
[For maintainers] Suggested jobs to run (before merge) run-slow: gemma4_unified |
|
CI Dashboard: View test results in Grafana |
5 participants
Address issues uncovered with weights conversion scripts.
Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
@ArthurZucker @molbap