Avner/sglang fa4 phnx - #9
Conversation
…d sent to draft process
… flashinfer dependency
…th a mask tensor can be loaded properly
Strips Phoenix V1 support (PhoenixLlamaForCausalLM, use_phoenix config flag, --phoenix CLI flags, PHOENIX_70B paths, use_eagle_or_phoenix abstraction, phoenix-specific activation conditioning branches, mask_tensor loader skip). Preserves all other improvements from avner/sglang-fa4-phnx. The companion branch avner/sglang-fa4-phnx differs from this branch solely by the phoenix code removed here. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This reverts commit 640b22c.
![arnica-github-connector[bot]](https://avatars.githubusercontent.com/in/144692?s=60&v=4){kind=small}
| self.draft_hf_config = AutoConfig.from_pretrained(draft) | ||
| self.max_model_len = min( | ||
| self.max_model_len, self.draft_hf_config.max_position_embeddings) | ||
| self.draft_hf_config = AutoConfig.from_pretrained(draft, trust_remote_code=True) |
There was a problem hiding this comment.
Static Code Analysis Risk: Together python huggingface trust remote code
trust_remote_code=True downloads and executes arbitrary Python code from the model repository without sandboxing (OWASP LLM03:2025 Supply Chain). A malicious or compromised model repo can achieve RCE on every host that loads the model (CWE-94). Pin to a verified commit hash and audit remote code before use, or use models that don't require trust_remote_code.
Severity: High 🚨
Status: Open 🔴
References:
- https://cwe.mitre.org/data/definitions/94
- https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel.from_pretrained
- https://genai.owasp.org/llmrisk/llm032025-supply-chain/
- https://hiddenlayer.com/research/weaponizing-machine-learning-models-with-ransomware/
Suggested reviewers 🧐: @avnermay
More details:
If you see an issue, please contact Shasheen in the #security-engineering Slack channel.
Take action by replying with an [arnica] command 💬
Actions
Use [arnica] or [a] to interact with the Arnica bot to acknowledge or dismiss code risks.
To acknowledge the finding as a valid code risk: [arnica] ack <acknowledge additional details>
To dismiss the risk with a reason: [arnica] dismiss <fp|accept|capacity> <dismissal reason>
Examples
-
[arnica] ack This is a valid risk and I'm looking into it -
[arnica] dismiss fp Dismissed - Risk Not Accurate: (i.e. False Positive) -
[arnica] dismiss accept Dismiss - Risk Accepted: Allow the risk to exist in the system -
[arnica] dismiss capacity Dismiss - No Capacity: This will need to wait for a future sprint
| self.max_model_len, self.hf_config.max_position_embeddings) | ||
| if self.speculate: | ||
| assert 1 <= self.num_gpus <= 8 # this codebase only works on one node | ||
| self.hf_config = AutoConfig.from_pretrained(model, trust_remote_code=True) |
There was a problem hiding this comment.
Static Code Analysis Risk: Together python huggingface trust remote code
trust_remote_code=True downloads and executes arbitrary Python code from the model repository without sandboxing (OWASP LLM03:2025 Supply Chain). A malicious or compromised model repo can achieve RCE on every host that loads the model (CWE-94). Pin to a verified commit hash and audit remote code before use, or use models that don't require trust_remote_code.
Severity: High 🚨
Status: Open 🔴
References:
- https://cwe.mitre.org/data/definitions/94
- https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel.from_pretrained
- https://genai.owasp.org/llmrisk/llm032025-supply-chain/
- https://hiddenlayer.com/research/weaponizing-machine-learning-models-with-ransomware/
Suggested reviewers 🧐: @avnermay
More details:
If you see an issue, please contact Shasheen in the #security-engineering Slack channel.
Take action by replying with an [arnica] command 💬
Actions
Use [arnica] or [a] to interact with the Arnica bot to acknowledge or dismiss code risks.
To acknowledge the finding as a valid code risk: [arnica] ack <acknowledge additional details>
To dismiss the risk with a reason: [arnica] dismiss <fp|accept|capacity> <dismissal reason>
Examples
-
[arnica] ack This is a valid risk and I'm looking into it -
[arnica] dismiss fp Dismissed - Risk Not Accurate: (i.e. False Positive) -
[arnica] dismiss accept Dismiss - Risk Accepted: Allow the risk to exist in the system -
[arnica] dismiss capacity Dismiss - No Capacity: This will need to wait for a future sprint
1 participant
No description provided.