Published material
I have started experimenting with a desktop application that lets me build and train my own small language model from scratch. I want to understand the whole process visually, especially what each setting is really changing and how the different stages fit together. The attached field guide maps the path from licensed text and a custom tokenizer through random weights, pre-training, checkpoints, sampling, chat fine-tuning, and evaluation. This is still an experiment and a way for me to make the system less abstract; I am not claiming that the model has finished training or produced results yet.
Loading published artifact…
↧building-a-small-language-model.html27 KB · published source