Bonsai Demo

[ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization

Generate images locally

[NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization

OpenSSA: Small Specialist Agents based on Domain-Aware Neurosymbolic Agent (DANA) architecture for industrial problem-solving

Downscaling Intelligence: Exploring Perception and Reasoning Bottlenecks in Small Multimodal Models

Coding agent built around small (<10B) models for local development with a Rust (Actix Web, Diesel)/Typescript (Solid, Tailwind) opinionated stack. No coding knowledge needed. Multiple agents and roles - Product Owner, PM, Engineering Manager, Rust Engineer, SolidJS Engineer, etc. work together to build full-stack apps.

[CVPR 2024] Asymmetric Masked Distillation for Pre-Training Small Foundation Models

This repository features a custom-built decoder-only language model (LLM) with a total of 37 million parameters 🔥. I train the model to be able to ask question from a given context

BobVLM – A 1.5B multimodal model built from scratch and pre-trained on a single P100 GPU capable of image descriptions and moderate question answering. 🤗🎉

Overview of self-supervised learning of tiny models, including distillation-based methods (aks. self-supervised distillation) and non-distillation methods.

Code for "On the Surprising Efficacy of Distillation as an Alternative to Pre-Training Small Models"

Provide specialized AI agents that develop, review, debug, and deploy production-ready code efficiently across various programming tasks.

Open-source harness distillation: frontier teachers improve prompts, tools, validators, skills, and runtime policies around weaker models.

Help us define the Pareto front of small models for MNIST classification. Frugal AI.

If OpenAI builds engines, VOID builds the chassis.

Tiny local AI agent for edge devices, built to make small models useful with workflows, personas, file/web context, and safe tool-like actions.

Manage Hermes CLI skins in a web UI, edit live YAML fields, and save or activate skins with built-in previews.

Fine-tunes a T5-small model on the TellMeWhy dataset using context injection from a large language model (Gemini) to improve causal reasoning for “why” questions in narratives. Combines efficient training with human and automated evaluations to assess impact.

quickthink is a local-first CLI and Python library that wraps Ollama-backed LLM calls with a compressed plan-then-answer scaffold and latency-aware routing. It adds a short validated planning step for multi-step prompts and routes simple ones straight through. Local inference control for small models.