A novel teacher‑free distillation method that replaces live teacher inference with compact precomputed random projection teacher fingerprints, enabling storage-efficient, high-performance knowledge transfer and enhanced student training.

Official Repo of OpenArchX Framework.

Layer-adaptive quantization of diffusion models using Fisher Information (LAQFI), presented at IEIE 2024.

O(N) Linear Complexity Transformer - 125x faster than standard attention using cumulative sum trick. Pure PyTorch implementation.

Notes and Labs from MIT 6.5940, (Fall 2023) lecture.

First 1-bit (BitNet b1.58) recursive reasoner for Sudoku-Extreme - distilled from a 7M-param FP TRM teacher into a 1.4 MB ternary student