Neural Computers (The Data Pipeline)

📘 Blog   ·   🎬 Demo Gallery   ·   📄 Paper

News

• 2026 April: First open-source release of Neural Computers, starting with the data pipeline for CLI and GUI trajectory generation.

Abstract

Neural Computers (NCs) are neural networks that unify computation, memory, and I/O in a single latent runtime state. The long-term goal is the Completely Neural Computer (CNC): a general-purpose neural computer that challenges the layered architecture of conventional computers. As an initial step, this project studies whether executable dynamics can be learned solely from collected I/O traces, without access to instrumented program state, using CLI and GUI trajectories as training and evaluation data.

Quick Start

Environment Python 3.9+ · Docker · asciinema · agg · ffmpeg

1. CLI / Asciinema

python3 main.py cligen asciinema record --command "python3 --version"
python3 main.py cligen asciinema cast-to-mp4 workspace/cligen_general/casts

2. CLI / VHS

python3 main.py cligen vhs build-image
python3 main.py cligen vhs generate-basic --count 10 --prefix demo
python3 main.py cligen vhs make-manifest
python3 main.py cligen vhs run-manifest

3. GUI / Synthetic

pip install -r engine/gui/synthetic_data_collection/requirements.txt
python3 main.py guiworld build-image
python3 main.py guiworld synthetic --count 1 --max-workers 1

Runtime Generations from Neural Computers

_{Using training data generated by the code released in this repository.}

CLIGen (general / asciinema)

CLIGen (clean / vhs)

GUIWorld (GUI)

_{Alternating: Conventional Computer / Neural Computer}

Acknowledge

Our models (not released here) are built from Wan2.1 and Matrix-Game-2. The data engine for CLIGen (General) is built from Asciinema, the data engine for CLIGen (Clean) is from VHS, the data engine for GUIWorld (Random) is modified directly from Neural-OS, and the data engine for GUIWorld (CUA) is built according to Claude CUA.

Data Engine	Source
CLIGen (General)	Asciinema
CLIGen (Clean)	VHS
GUIWorld (Random)	Neural-OS
GUIWorld (CUA)	Claude CUA