Jakub
67061cf373
Document six completed LR runs as preparation for full Aya training on Perun. Record selection of LR 0.00010 by lowest final validation loss, selected-model lm-eval and partial SK-bench results, reproducible provenance, and dataset-access/context-truncation limitations. Preserve existing 30k and Titan material.
976 B
976 B
| 1 | learning_rate | run | global_step | epoch | final_eval_step | final_eval_loss | min_eval_step | min_eval_loss | max_grad_norm | p95_grad_norm | train_runtime | eval_observations | gradient_observations |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2 | 8e-05 | lr_0p00008 | 18688 | 1.0 | 18688 | 0.875216543674469 | 18688 | 0.875216543674469 | 9.149866104125977 | 1.5837211608886719 | 79243.6449 | 19 | 1868 |
| 3 | 0.0001 | lr_0p00010 | 18688 | 1.0 | 18688 | 0.87261962890625 | 18688 | 0.87261962890625 | 8.578277587890625 | 1.6475107669830322 | 79344.5851 | 19 | 1868 |
| 4 | 0.00015 | lr_0p00015 | 18688 | 1.0 | 18688 | 0.8842697143554688 | 18688 | 0.8842697143554688 | 28.051427841186523 | 1.9590225219726562 | 80127.6428 | 19 | 1868 |
| 5 | 0.00018 | lr_0p00018 | 18688 | 1.0 | 18688 | 0.8914602994918823 | 18000 | 0.891387939453125 | 6.5584211349487305 | 2.163313627243042 | 79892.0077 | 19 | 1868 |
| 6 | 0.0002 | lr_0p00020 | 18688 | 1.0 | 18688 | 0.8976690173149109 | 18000 | 0.897506594657898 | 10.969125747680664 | 2.306173801422119 | 79710.3252 | 19 | 1868 |
| 7 | 0.00023 | lr_0p00023 | 18688 | 1.0 | 18688 | 0.9032362699508667 | 18000 | 0.9031931161880493 | 23.10581398010254 | 2.501060724258423 | 79789.3562 | 19 | 1868 |