Back-to-basics language model

A 71.5M-parameter transformer trained from scratch on English Wikipedia. It runs entirely in your browser on the CPU (WebAssembly); nothing is sent to a server.

Article title

Opening text (optional, steers the topic)

Sampling settings

Temperature

Top-k

Top-p

Repetition penalty

Max tokens

Seed (blank = random)

Loading model...

The model has learned Wikipedia's style and structure, but at this size almost all specific facts (dates, names, numbers) are invented. Temperature 0 gives deterministic greedy output.