Joe Rice-Jones at XDA Developers spent a week training a ChatGPT-style model from scratch on Nvidia DGX Spark mini workstations. His result is that fewer bits and newer tooling did not make training faster. He used Andrej Karpathy's nanochat and compared three ways of running the training math...
