Official PyTorch implementation of "GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance" (ICML 2025)
-
Updated
Apr 13, 2026 - Python
#
llm-quantization
Here are 21 public repositories matching this topic...
Open quantization tooling for TurboQuant-style low-bit LLM releases, stock GGUF deployment, and Apple Silicon runtime experiments.
apple toolkit tooling tensor quantization apple-silicon llm llm-inference gguf llm-quantization turboquant
-
Updated
May 5, 2026 - Python
[CAAI AIR'24] Minimize Quantization Output Error with Bias Compensation
post-training-quantization llm-compression output-error-optimization bias-compensation llm-quantization
-
Updated
Mar 12, 2025 - Python
A high-performance, memory-efficient healthcare framework that deploys fine-tuned Large Language Models (LLMs) on edge devices. Multi-agent system to provide personalized diagnostic reasoning, health education, and dietary planning.
-
Updated
Sep 7, 2025 - Jupyter Notebook
A more deep research about TurboQuant algorithms
-
Updated
Apr 6, 2026 - Python
Ternary Quantization for LLMs: Implement balanced ternary (T3_K) weights for 2.63-bit quantization—the first working solution for modern large language models.
-
Updated
Nov 29, 2025 - C++
Let me make GGUF files quickly
-
Updated
Jun 4, 2025 - Python
Production-grade LLM quantization, benchmarking, and edge deployment toolkit. Supports bitsandbytes INT8/INT4, GPTQ (Hessian calibration), AWQ (activation-aware), and GGUF (Q2_K–Q8_0). Four-dimensional benchmarking: perplexity, TPS/TTFT, VRAM profiling, and LLM-as-Judge quality scoring. RTX 5090 Blackwell sm_120 ready.
python benchmarking transformers pytorch nvidia-cuda model-compression onnx edge-ai huggingface streamlit blackwell post-training-quantization awq llama-cpp gptq bitsandbytes ollama gguf llm-quantization rtx-5090
-
Updated
Jun 14, 2026 - Python
Paired capability-level GGUF quantization fragility benchmark across Qwen2.5-3B and SmolLM2 1.7B.
-
Updated
Jun 25, 2026 - Python
LLM quantization project built around `llama.cpp` + `Ollama` + `GGUF`
-
Updated
Mar 22, 2026 - Python
Local & lightweight LLM inference runtime in C++ with support for GGUF & quantization
open-source library opensource cpp17 mit-license cpp-library cpp-lib llm cpp-module llm-inference llm-local llm-tools llm-framework gguf llm-library llm-quantization llm-integration lightweight-llm
-
Updated
Feb 27, 2026
GWIQ-Atlas: is a brain-atlasing and model-interpretability suite that combines per-layer census, compliance behaviour tracing, SAE features, and quantization analyses for LLMs.
python transformers quantization sae sparse-autoencoder brain-atlas huggingface activation-analysis mechanistic-interpretability llm-analysis llm-quantization llm-interpretability model-atlas feature-census
-
Updated
Jul 5, 2026 - Python
Shift-based post-training quantization analysis for LLMs (ShiftQuant paper)
-
Updated
Mar 28, 2026 - Python
Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recovery
loop-detection 2-bit chain-of-thought llm-inference llm-quantization hybrid-inference low-bit-reasoning quantized-reasoning reasoning-acceleration
-
Updated
Jul 3, 2026 - Python
OpenVINO Model Manager — desktop GUI for Intel Arc
linux-gui openvino intel-gpu nncf pyqt6 intel-arc llm-tools optimum-intel llm-quantization llm-conversion
-
Updated
Jun 22, 2026 - Python
A high-performance inference engine optimized for deploying quantized LLMs on edge devices. Focuses on SIMD optimizations and memory management.
-
Updated
Apr 9, 2026 - C++
pytorch quantization dequantization 2-bit hugging-face hugging-face-hub llm-quantization torch-quantization
-
Updated
Jun 26, 2024 - Jupyter Notebook
PentaNet extends BitNet's ternary quantization to pentanary {-2,-1,0,+1,+2}, improving perplexity by 6.4% at 124M params while preserving zero-multiplier arithmetic.
-
Updated
Apr 17, 2026 - Python
Implemented and fine-tuned BERT for a custom sequence classification task, leveraging LoRA adapters for efficient parameter updates and 4-bit quantization to optimize performance and resource utilization.
nlp lora quantization data-augmentation nlp-augmentation llm qlora llm-fine-tuning peft-fine-tuning-llm llm-quantization
-
Updated
Dec 30, 2024 - Jupyter Notebook
Improve this page
Add a description, image, and links to the llm-quantization topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the llm-quantization topic, visit your repo's landing page and select "manage topics."