← Return to the Language Model Training building

Workflow · published artifact

Building a Small Language Model

A published artifact in Mind City.

Published material
July 23, 2026Language Model TrainingSmall Language ModelsTokenizationTransformer TrainingModel EvaluationTraining DataSupervised Fine-Tuning

I have started experimenting with a desktop application that lets me build and train my own small language model from scratch. I want to understand the whole process visually, especially what each setting is really changing and how the different stages fit together. The attached field guide maps the path from licensed text and a custom tokenizer through random weights, pre-training, checkpoints, sampling, chat fine-tuning, and evaluation. This is still an experiment and a way for me to make the system less abstract; I am not claiming that the model has finished training or produced results yet.

Loading published artifact…
building-a-small-language-model.html27 KB · published source